{"id":"c987f871-3016-4567-b8dd-4bbad3d81357","arxiv_id":"2505.02867","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A zero-shot referring expression segmentation method that uses attribute prompting to reason about object parts and implicit descriptions, outperforming prior zero-shot and several fine-tuned baselines.","lead":"RESAnything is a training-free system that locates the image region described by free-form text, including object parts and descriptions of material, function, or style. It asks a vision-language model to describe candidate regions and compare those descriptions to the query, outperforming prior zero-shot methods and several fine-tuned baselines.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim presupposes that SAM's proposal stage always contains a usable mask for the target region; the paper's own conclusion concedes this can fail, and the ABO benchmark's SAM-derived GT cannot detect such failures.","rationale":"I agree with the reader's weakest assumption: proposal coverage is the load-bearing premise. The reader's CONDITIONAL verdict is appropriate; I would not change it. The concern is internal to the method, not a disagreement with community consensus: Section 3.1's fixed proposal set is an upper bound on what any subsequent selection can output. The paper's own limitation statement in Section 5 and the SAM-based annotation protocol in Section 8.3 both corroborate the risk. The concrete oracle test addresses the concern directly and is cheap to run. Additional issues (unreleased code, no variance bars) are real but secondary; they affect reproducibility, not the internal logic. The benchmark bias on ABO-Image-ARES is significant but does not by itself decide the external-benchmark claims; this is why the oracle test on RefCOCO/ReasonSeg is the right check.","tokens_in":28205,"tokens_out":5130,"duration_ms":51866,"concrete_test":"Compute an oracle upper bound for the proposal stage: run the exact SAM configuration from Section 3.1 on all images of RefCOCO val and ReasonSeg val, and for each ground-truth mask compute the maximum IoU over (a) individual proposals and (b) unions of up to five proposals, matching Algorithm 1's grouping behavior. Report the percentage of queries whose oracle IoU is below 0.5, split by object-level vs part-level queries. If that percentage is nontrivial (e.g., >10%) for part-level queries, proposal coverage is the bottleneck and the claim of arbitrary part-level RES is unsupported; if it is near zero, the reported performance is governed by the selection stage and this objection is settled.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline result—that a training-free pipeline beats fine-tuned reasoning segmenters on part-level and implicit queries—depends on a single premise: for every query, the fixed proposal set M from Section 3.1 contains some mask (or union of masks) with nontrivial IoU against the target. RESAnything only selects among SAM proposals; it cannot recover a region that was never proposed. The proposal configuration intentionally removes small masks (segments below 0.1% of image area are filtered) and samples only sparse points (0.015% of pixels), so small parts—exactly the targets emphasized for arbitrary RES—are the most likely to be absent. The paper explicitly concedes this in Section 5: 'SAM occasionally fails to produce the best mask candidates, potentially degrading RES accuracy'. The weakness is hidden on the new ABO-Image-ARES benchmark because its ground-truth masks are generated by SAM itself (Section 8.3), so the evaluation shares the proposal generator and cannot expose coverage gaps. On RefCOCO and ReasonSeg, the oracle ceiling of the proposal stage has not been reported, so the large gains (68.5 cIoU on RefCOCO val; 74.6/72.5 gIoU/cIoU on ReasonSeg) could reflect favorable SAM coverage for those datasets rather than a robust arbitrary-segmentation capability.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"RESAnything is a training-free, open-vocabulary referring expression segmentation pipeline. Given an image and a free-form expression, it first generates SAM mask proposals, then uses an MLLM to produce an attribute-rich reference text for the target and candidate texts for each proposal using visual prompts (mask crop and bounding box). A multi-metric selection stage combines MLLM binary decisions and CLIP scalar scores to pick the final mask or return a textual explanation. The paper reports large gains over existing zero-shot methods on RefCOCO/+/g, state-of-the-art results on ReasonSeg, and introduces a new ABO-Image-ARES benchmark for part-level and implicit queries.","tokens_in":28464,"tokens_out":6137,"duration_ms":62538,"significance":"If the reported results hold, this is a meaningful advance: RESAnything is the first LLM-based, zero-shot RES method, requires no fine-tuning, and uses fixed hyperparameters, which makes the comparison against trained systems clean and reproducible in principle. The external human-annotated benchmarks (RefCOCO/+/g and ReasonSeg) support the core claim that careful attribute prompting plus SAM proposals can outperform prior zero-shot methods by a wide margin and rival supervised systems on vanilla RES. The visual-prompt ablation (Table 4), MLLM-backbone comparison (Table 5), and selector ablation (Supplementary Table 6) are useful and strengthen the paper. The main weaknesses are the unmeasured proposal-coverage ceiling introduced in Section 3.1 and the SAM-derived ground truth of the new ABO-Image-ARES benchmark (Section 8.3), which together weaken the broader claim about arbitrary part-level and implicit queries.","major_comments":[{"comment":"The central 'arbitrary referring segmentation' claim is bounded by the quality of the SAM proposal stage: RESAnything can only select among masks in M, generated with point sampling at 0.015% of pixels and filtering segments smaller than 0.1% of image area. The paper does not report the oracle ceiling of this proposal stage (e.g., the best IoU achievable from M, or from unions of masks in M, per query) on RefCOCO, ReasonSeg, or ABO-Image-ARES. Since Section 5 itself concedes that 'SAM occasionally fails to produce the best mask candidates,' the large reported gains (68.5 cIoU on RefCOCO val; 74.6/72.5 gIoU/cIoU on ReasonSeg) could be partly an artifact of favorable SAM coverage on these datasets rather than a robust capability on arbitrary part-level queries. Please add an oracle-coverage analysis and a failure analysis for cases where no proposal or union reaches a meaningful IoU.","section":"Section 3.1 and Section 5"},{"comment":"The new ABO-Image-ARES benchmark is not an independent test of the central claim because its ground-truth masks are themselves SAM outputs: annotators selected GT regions from SAM's automatic candidates, with manual refinement via SAM interactive mode. Since RESAnything's selection is restricted to SAM proposals, the benchmark shares the proposal generator with the method and cannot expose cases where the correct region is absent from M. The large margins over LISA and GLaMM on ABO (78.2/72.4 vs 46.2/38.7 gIoU/cIoU) are therefore not a sound basis for the claim of 'significantly outperforming existing methods on challenging scenarios involving implicit queries and complex part-level relations.' This issue should be fixed by re-annotating a held-out subset with human polygon labels independent of SAM, or by removing ABO from the paper's central claims and relying on external human-annotated benchmarks.","section":"Section 8.3 / Table 3"},{"comment":"The abstract claims 'significantly outperforms existing methods on challenging scenarios involving implicit queries and complex part-level relations,' but on the part-only RefCOCOm benchmark in Supplementary Table 8, RESAnything (27.6/26.5/25.8) is below supervised GLaMM (30.0/27.2/31.8) and only modestly above CaR (10.9/10.6/10.9). Please qualify the claim or explain the discrepancy, since it bears directly on the part-level superiority stated as a headline contribution.","section":"Abstract / Supplementary Table 8"}],"minor_comments":[{"comment":"The text says 'We report gIOU for RefCOCO, RefCOCO+, and RefCOCOg,' but Table 1's header says the results are 'reported as cIoU values.' Please clarify which metric is actually reported.","section":"Section 4 / Table 1"},{"comment":"The final CLIP threshold in Algorithm 1 is set to 1 for all experiments, yet the supplementary (Section 6.2) reports that text-to-image CLIP scores remain below 0.3; please justify this threshold or calibrate it, and report sensitivity to it.","section":"Algorithm 1 / Section 3.2"},{"comment":"The visual-prompt ablation table is difficult to read because the row configurations are not explicitly labeled; add clear row labels indicating which prompts (image, mask-cropped, bounding box, contour, blur) are active in each row.","section":"Table 4"},{"comment":"Please provide a URL for the ABO-Image-ARES dataset and for the implementation, as the paper currently gives no release mechanism despite claiming these as contributions.","section":"Reproducibility"},{"comment":"There are minor typos and textual errors, e.g., 'CLIP sores' in Section 3.2 and 'tehcamera' in Figure 16; please proofread the manuscript.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope and the external benchmark results are substantial enough that I do not see grounds for rejection. The main risk is that the ABO benchmark's SAM-derived ground truth, combined with the unmeasured proposal-coverage ceiling, inflates the part-level and implicit-query claims. I would request the oracle-coverage analysis and either an independent human re-annotation of an ABO subset or a narrowing of the paper's central claims. No concerns about citation patterns or novelty disclosure beyond the missing code/data availability."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know. First, the attribute prompting idea is genuinely new relative to the cited baselines and it works: on RefCOCO/+/g it roughly doubles the best training-free zero-shot results, and on ReasonSeg it beats fine-tuned LISA-13B and SAM4MLLM by a wide margin. Second, the paper's own limitation section concedes the load-bearing weakness: the system can only select from SAM's proposals, and SAM occasionally fails to propose the right mask, especially for small parts that the pipeline explicitly filters out.\n\nThe method is simple, training-free, and clearly described. The ablations on visual prompts and MLLM backbones are honest and informative, and reporting eight averaged runs shows care, though no variance or significance tests are given. The external benchmark numbers are the real evidence; nothing about the core pipeline is circular on RefCOCO or ReasonSeg.\n\nThe soft spot is the ABO-Image-ARES benchmark. Ground-truth masks were selected from SAM's automatic proposals, and the method also selects from SAM proposals, so the shared proposal generator means the 78/72 gIoU/cIoU numbers there likely overstate capability. That is a benchmark design flaw, not a fatal flaw in the method, but it should be fixed by re-annotating with human-drawn or independently produced masks. More importantly, the paper reports no oracle ceiling for the proposal stage on any benchmark, so we do not know how much of the remaining error is proposal coverage versus selection reasoning. Code and data are not released, which matters for a training-free method that should be easy to reproduce.\n\nNone of this undermines the central contribution. The attribute prompting idea is worth building on, and the ReasonSeg results stand on human-annotated ground truth. This deserves a serious referee. The revision requests are concrete: re-annotate or relabel the ABO benchmark, report proposal-oracle upper bounds, release code and data, and add variance estimates. Those are revision requests, not grounds for rejection.","headline":"Attribute prompting is a real and effective new idea for zero-shot referring segmentation, but the method's ceiling is set by SAM's proposal coverage, and the new benchmark's SAM-derived GT inflates the headline numbers there.","tokens_in":28998,"tokens_out":1791,"would_cite":true,"duration_ms":19626,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"RESAnything claims that a training-free, LLM-driven pipeline can segment arbitrary objects, parts, and implicit references, outperforming fine-tuned reasoning segmentation models.","keywords":["referring expression segmentation","zero-shot segmentation","attribute prompting","chain-of-thought reasoning","multimodal large language models","SAM proposals","part-level segmentation","ABO-Image-ARES"],"falsifier":"Measure oracle recall of SAM's automatic proposals against the ground-truth masks of ABO-Image-ARES: if a substantial fraction of targets have no proposal with IoU above, say, 0.5, then RESAnything cannot reach high accuracy on those queries; constructing a part-level benchmark where this recall failure occurs would refute the claim that the pipeline handles arbitrary part-level references.","tokens_in":27983,"feed_emoji":"🎯","tokens_out":6398,"duration_ms":59547,"temperature":0.7,"pith_summary":"This paper argues that arbitrary referring expression segmentation—finding the mask for an object, a part, or an implicitly described region such as 'the thing that keeps your neck warm'—can be solved with no training at all. Its method, RESAnything, first asks SAM to propose candidate masks, then asks a multimodal LLM to write attribute-level sentences about the query and about every proposal, and finally selects the best proposal by combining LLM yes/no decisions with CLIP scores. On standard RES benchmarks this training-free recipe roughly doubles the best prior zero-shot results (68.5 vs 33.6 cIoU on RefCOCO val) and on ReasonSeg it surpasses fine-tuned methods such as SAM4MLLM by about 16 points. The same pipeline outperforms fine-tuned models by more than 30 points on the paper's new part-level benchmark. If these results hold, arbitrary part and attribute queries become addressable in applications like e-commerce and robotics without collecting new segmentation annotations.","feed_headline":"Zero-shot pipeline beats fine-tuned models at referring segmentation","feed_subtitle":"An LLM describing SAM masks as attributes picks parts and implicit objects with no training, doubling prior zero-shot scores.","key_machinery":"The load-bearing mechanism is attribute prompting: asking an MLLM to convert the referring expression into a single sentence rich in visual attributes (shape, color, location, function, and relation to surroundings), and to generate similarly structured candidate sentences for every SAM proposal, using a mask-cropped view for appearance and a bounding-box view for context. This turns the hard image-to-expression matching problem into a text-to-text comparison problem along with a secondary text-to-image check. The selection stage then combines four metrics—MLLM binary decisions for text-to-text and text-to-image, plus CLIP scores for both—and adds the union of positively judged masks as a candidate, which lets a single semantic entity that spans several segments (like all the legs of a chair) be returned as one mask.","core_discovery":"The central claim is that a two-stage, fully training-free pipeline can segment what previous RES systems cannot: parts of objects, and regions referred to only by function, design, style, material, or packaging text. In the first stage, SAM generates and filters mask proposals, and an MLLM is prompted to produce a reference text describing the query's visual attributes and a candidate text for each proposal using mask-cropped and bounding-box views. In the second stage, the method selects the proposal whose candidate text best matches the reference text, using MLLM binary yes/no judgments as the primary filter and CLIP scalar scores to break ties and to combine multiple masks into one target. The paper reports 68.5 cIoU on RefCOCO val, 74.6/72.5 gIoU/cIoU on ReasonSeg, and 78.2/72.4 on ABO-Image-ARES, all without training or fine-tuning.","pith_inferences":["Beyond the paper: because the authors' own limitation analysis identifies SAM proposal recall as the ceiling, a natural next test is to replace automatic SAM proposals with denser proposal generators or an iterative 'zoom in' loop that re-proposes around the top candidates; the selection stage should transfer unchanged.","Beyond the paper: attribute prompting converts region matching into text matching, so the same two-stage idea could be applied to other fine-grained vision-language tasks such as referring detection, visual grounding with negative queries, or part-level VQA, by inserting attribute descriptions before any matching step.","Beyond the paper: the observation that MLLM binary decisions beat scalar scores suggests that for many MLLM-based selection tasks, forcing a categorical yes/no with a follow-up ranker may be more reliable than asking for continuous scores, which is testable independently of segmentation.","Beyond the paper: the benchmark's construction from product bulletpoints through an LLM could be repeated on other catalog data to test whether the approach extends beyond the original imagery, e.g., fashion or furniture catalogs with different part vocabularies."],"forward_implications":["If the central claim holds, referring segmentation no longer needs task-specific training data; any new domain only requires SAM to propose masks and an MLLM to describe them.","Reasoning-style queries such as 'what would keep a dog restrained in a hallway' become solvable by attribute description and text matching rather than by fine-tuning on curated reasoning masks.","The union-of-positive-masks rule extends the method to multi-part targets and occlusion cases, such as all legs of a sofa or a person partially hidden.","Because the pipeline is modular, swapping the MLLM (Pixtral, Qwen2-VL, Claude) changes accuracy only slightly, suggesting the recipe transfers across backbones.","The ABO-Image-ARES benchmark gives the community a way to measure part-level, material/function/style queries that prior RES datasets did not isolate."],"supporting_citations":[{"why":"Supplies the automatic mask proposals that the entire selection stage operates on.","marker":"[25]"},{"why":"Provides the MLLM used for reference and candidate text generation and for binary decisions.","marker":"[4]"},{"why":"Supplies CLIP scalar scores used to disambiguate and combine candidates.","marker":"[44]"},{"why":"Defines the reasoning segmentation task and serves as the main fine-tuned baseline on ReasonSeg.","marker":"[26]"},{"why":"Is the strongest prior training-free zero-shot RES baseline that RESAnything compares against.","marker":"[52]"},{"why":"Serves as the pixel-grounding MLLM baseline on ReasonSeg and ABO-Image-ARES.","marker":"[45]"},{"why":"Is the strongest supervised baseline on ReasonSeg and standard RES benchmarks.","marker":"[11]"},{"why":"Provides the product images and metadata from which the new ABO-Image-ARES benchmark is built.","marker":"[13]"}],"fun_headline_variants":["Zero-shot LLM attribute prompting segments object parts implicitly","RESAnything attribute prompting for zero-shot part segmentation","LLM attribute prompts let SAM find parts without any training","Zero-shot referring segmentation by prompting SAM with LLM attributes","Training-free attribute prompting for arbitrary referring expressions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's ceiling is set by SAM's automatic proposals: if the region the expression refers to is not among the proposed masks, no attribute reasoning can recover it, a limitation the paper itself acknowledges.","fun_headline_variants_meta":{"raw":{"variants":["Zero-shot LLM attribute prompting segments object parts implicitly","RESAnything attribute prompting for zero-shot part segmentation","LLM attribute prompts let SAM find parts without any training","Zero-shot referring segmentation by prompting SAM with LLM attributes","Training-free attribute prompting for arbitrary referring expressions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001119,"raw_usage":{"total_tokens":4663,"prompt_tokens":955,"completion_tokens":3708,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":571,"completion_tokens_details":{"reasoning_tokens":3633}},"tokens_in":571,"tokens_out":3708,"duration_ms":24251,"temperature":1.0,"reasoning_tokens":3633,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:08:36.419527+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure oracle recall of SAM's automatic proposals against the ground-truth masks of ABO-Image-ARES: if a substantial fraction of targets have no proposal with IoU above, say, 0.5, then RESAnything cannot reach high accuracy on those queries; constructing a part-level benchmark where this recall failure occurs would refute the claim that the pipeline handles arbitrary part-level references.","supporting_citations":[{"cited_title":"Segment any- thing","cited_arxiv_id":null,"evidence_quote":"Supplies the automatic mask proposals that the entire selection stage operates on."},{"cited_title":"Lisa: Reasoning segmentation via large language model","cited_arxiv_id":null,"evidence_quote":"Defines the reasoning segmentation task and serves as the main fine-tuned baseline on ReasonSeg."},{"cited_title":"Clip as rnn: Segment countless visual concepts without training endeavor","cited_arxiv_id":null,"evidence_quote":"Is the strongest prior training-free zero-shot RES baseline that RESAnything compares against."},{"cited_title":"Glamm: Pixel grounding large multimodal model","cited_arxiv_id":null,"evidence_quote":"Serves as the pixel-grounding MLLM baseline on ReasonSeg and ABO-Image-ARES."},{"cited_title":"Abo: Dataset and benchmarks for real-world 3d object understand- ing","cited_arxiv_id":null,"evidence_quote":"Provides the product images and metadata from which the new ABO-Image-ARES benchmark is built."}],"review_version":1}