{"id":"b1c31b1d-cf29-4703-a54f-05b2c1202602","arxiv_id":"2608.02791","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"STAMPlus extends All-Mask Prediction to multi-target segmentation with a structured target list bound to a shared multi-class mask space, delivering state-of-the-art accuracy with single-pass efficiency.","lead":"STAMPlus is a new method that lets a multimodal language model segment many objects at once by first writing a structured list of targets, then classifying all image patches in one non-autoregressive pass. It reports top scores on referring, reasoning, open-vocabulary, instance, and remote-sensing segmentation while keeping dialogue ability and cutting multi-target latency to about 5 seconds.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Phase 1 list quality is the load-bearing bottleneck: Table VIII(a) shows human-provided boxes raise cIoU from 77.4 to 89.1, yet the multi-target SOTA claims are reported end-to-end without Phase 1 list-quality ablations.","rationale":"I read the paper as a systematic extension of STAMP with real positive features: the controlled resolution scaling in Table XI, the honest ablations of shared components in Table X, and the unified-checkpoint design. The core risk is not the end-to-end numbers themselves but the attribution of the capability: the structured binding is evaluated only jointly with the structured generator. Table VIII(a) is the paper's own evidence that Phase 1 quality matters enormously, and the paper's proposed future work (RL or localization tools) confirms that the authors see this headroom. In the multi-target and open-vocabulary settings, a Phase 1 omission is unrecoverable because the mask head only scores the active set K. This makes the reader's weakest assumption load-bearing. The oracle-Phase-1 check would settle whether the bottleneck is the binding mechanism or the list generator. If the oracle delta is small, the division of labor between the two phases is healthy; if it is large, the paper should be scoped to say that accuracy is currently limited by the autoregressive target list, which changes the strength of the trilemma-resolution claim. I do not see a reason to move away from the reader's CONDITIONAL verdict; the concern reinforces it. The completeness of the claimed codebase and the controlled ablations count in the paper's favor, but they do not substitute for isolating Phase 1 list quality on the multi-target benchmarks.","tokens_in":26765,"tokens_out":9591,"duration_ms":92414,"concrete_test":"Re-evaluate the unified STAMPlus-7B checkpoint on MUSE and ADE20K-150 with a controlled oracle intervention: keep Phase 2 and the mask-token mechanism exactly as trained, but replace the model-generated Phase 1 JSON list with the ground-truth target list (labels, IDs, and boxes where available) before <SEG>. Compare the resulting MUSE avg gIoU/cIoU and ADE20K mIoU against the reported end-to-end numbers. If the oracle-Phase-1 delta is smaller than the margin over Text4Seg++ on MUSE and over the specialized open-vocabulary baselines on ADE20K, the dynamic binding is robust and the concern does not land. If the delta exceeds those margins, the paper should report Phase 1 parse rate, target recall, and order-error statistics per benchmark, and the SOTA claims should be conditioned on Phase 1 accuracy.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that one unified STAMPlus checkpoint resolves the trilemma in broader settings (RefCOCO family, gRefCOCO, ReasonSeg, RRSIS-D/EarthReason, ADE20K/PC-59/PAS-20, MUSE) via structured all-mask prediction. For this to hold, the Phase 1 structured list must be parseable, complete, and correctly ordered for every target, because Phase 2's fixed 200-class head only assigns classes in the active ID set K defined by that list (Sec. III-B2). The paper's own Table VIII(a)/Sec. IV-D1 shows the binding is sharply sensitive to Phase 1 quality: replacing auto-generated target cues with human-provided boxes raises RefCOCO-family cIoU from 77.4 to 89.1, an 11.7-point jump that is larger than the reported margin over the best prior method. The text explicitly states that 'the quality of the Phase 1 target description has a substantial impact on Phase 2 segmentation' and proposes future RL/tool assistance; that admission is presented as an analysis, not as a limitation on the SOTA claim. Because the multi-target benchmarks (ADE20K, MUSE, RRSIS-D) are reported only end-to-end, the contribution of Phase 1 list accuracy is never isolated: a missing, duplicated, or misordered category/instance in Phase 1 is unrecoverable in Phase 2, so the reported mIoU/cIoU is an upper bound conditioned on the generative list. This is not a question of dishonest results; it is a question of whether the demonstrated capability belongs to the structured binding mechanism or to the autoregressive Phase 1 generator, and whether 'broader' holds when Phase 1 degrades.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper extends the authors' prior STAMP (binary all-mask prediction) to STAMPlus, which performs structured all-mask prediction for multi-target segmentation in MLLMs. STAMPlus lets Phase 1 autoregressively generate a JSON-style target list with explicit IDs and optional bounding boxes before the <SEG> trigger; Phase 2 then prefills image-aligned [MASK] tokens and, in a single non-autoregressive forward pass, classifies each token against a fixed-capacity multi-class head whose active IDs are dynamically bound to the Phase 1 list. A single unified checkpoint is evaluated across referring segmentation (RefCOCO family, gRefCOCO, ReasonSeg), remote-sensing small-target segmentation (RRSIS-D, EarthReason), open-vocabulary semantic segmentation (ADE20K, Pascal Context, Pascal VOC), instance-aware multi-target segmentation (MUSE), general multimodal instruction following, and latency scaling. The central claim is that STAMPlus resolves the accuracy–dialogue-compatibility–efficiency trilemma beyond single-target prediction, achieving state-of-the-art segmentation in these broader settings while retaining general dialogue ability and reducing 12-category latency from 13.50 s (repeated STAMP) to 5.16 s.","tokens_in":27110,"tokens_out":5159,"duration_ms":47979,"significance":"If the central claims hold, STAMPlus would be a valuable general interface for MLLM-based dense prediction: it extends a non-autoregressive, token-only-supervised mask prediction paradigm from binary targets to multi-category and multi-instance settings without task-specific fine-tuning. The paper has notable strengths: a single unified checkpoint is evaluated across many benchmarks; the resolution-scaling study in Table XI is a controlled comparison; the multi-target latency scaling in Fig. 9 is reported with mean and standard deviation; and the diagnostic experiments in Table VIII explicitly probe the interaction between Phase 1 and Phase 2. These controlled experiments and the breadth of evaluation are genuinely useful. The main uncertainty is whether the demonstrated multi-target capability is attributable to the structured binding mechanism or to the autoregressive Phase 1 list generator, because the paper's own Table VIII(a) shows a very large sensitivity to Phase 1 target-cue quality. This issue is load-bearing for the 'state-of-the-art across these settings' claim and must be addressed before the central claim can be accepted.","major_comments":[{"comment":"The paper's own analysis shows that the structured binding mechanism is sharply sensitive to the quality of the Phase 1 target list: replacing auto-generated target cues with human-provided boxes raises RefCOCO-family cIoU from 77.4 to 89.1, an 11.7-point gain that is larger than the reported margins over prior methods. Since Phase 2's active ID set K is defined entirely by the Phase 1 list (Sec. III-B2), a missing, duplicated, or misordered entry in Phase 1 is unrecoverable in Phase 2. The multi-target SOTA claims on ADE20K, MUSE, RRSIS-D, and EarthReason are reported only end-to-end, and no Phase 1 parse-success rate, list-completeness metric, or conditional result with an oracle/verified Phase 1 list is provided. The text acknowledges that 'the quality of the Phase 1 target description has a substantial impact on Phase 2 segmentation' and proposes future RL/tool assistance, but this admission is not carried into the SOTA claim as a limitation. The paper should report Phase 1 list-quality statistics and, where feasible, results conditioned on a human-verified or oracle Phase 1 list, so that the contribution of the structured binding mechanism can be separated from the autoregressive list generator. Without this, the central claim that the trilemma is resolved in broader settings is not fully supported.","section":"Sec. IV-D1, Table VIII(a), Sec. III-B2"},{"comment":"The abstract claims that 'Across these settings, STAMPlus achieves state-of-the-art segmentation performance,' but the MUSE results do not support this on the reported average metric: Text4Seg++ averages 63.8 while STAMPlus-7B averages 63.5 in Table VII(b). The body text appropriately says STAMPlus is 'competitive with Text4Seg++ overall' and exceeds it on the test split, but the abstract and contributions overstate the result. This is a claim-safety issue: either the abstract should be qualified, or the MUSE comparison should be presented with a metric or analysis that justifies the SOTA statement (for example, a significance test or a clear primary-metric argument).","section":"Abstract and Table VII(b)"},{"comment":"All accuracy numbers across the main comparison tables are reported without error bars, confidence intervals, or multiple-seed standard deviations; only the latency experiment in Fig. 9 includes mean and standard deviation. This is particularly important for the SOTA claims where margins are small (e.g., 63.5 vs. 63.8 on MUSE in Table VII(b), and 76.2 vs. 74.4 on RRSIS-D in Table V). Without any variance information, it is difficult for a reader to determine whether the reported differences are meaningful. The authors should report at least two or three seeds or, alternatively, explicitly state that results are single-run and avoid strong comparative wording where margins are within likely noise.","section":"Tables II–VII"}],"minor_comments":[{"comment":"In the READ row, '59.867.6' appears to be a typographical error for '59.8 67.6'; please fix the missing spacing.","section":"Table IV"},{"comment":"The high-resolution scaling description could be more precise about how the mask-token budget N is selected for different input resolutions and how the 1024–1280 and 2560–3200 ranges are determined; the controlled study in Table XI would be easier to interpret if the exact N values per benchmark were listed.","section":"Sec. III-B3"},{"comment":"The look-twice experiment uses attention, not predicted masks, to derive grounding regions; this is stated in the text, but the conclusion that 'segmentation-learned spatial grounding' supports downstream understanding is indirect. A one-sentence clarification that the mechanism tested is segmentation-induced attention, not mask-feature reuse, would reduce the risk of overreading.","section":"Sec. IV-D2, Table VIII(b)"},{"comment":"The paper states that 'The complete codebase ... is included in the supplementary material' but does not provide a public repository URL or commit hash. If this is a journal submission, please provide a link or state clearly in the final version how reviewers and readers can obtain the code.","section":"Introduction, Codebase"},{"comment":"The caption says 'marker size indicates model scale,' but the marker sizes are not otherwise defined; please add an explicit legend or note the correspondence between marker sizes and model scales (2B/7B/13B).","section":"Fig. 8"}],"recommendation":"major_revision","confidential_remarks":"The paper is a clear extension of the authors' CVPR 2026 STAMP work, and the self-citation is appropriate rather than problematic. The main risk is that the Phase 1 list-quality bottleneck, which the authors themselves demonstrate in Table VIII(a), is not treated as a limitation on the SOTA claim. If the authors add Phase 1 quality metrics and oracle-conditioned results, and revise the overbroad SOTA wording, the paper could become a strong contribution. The absence of a publicly accessible code link should also be checked during production; the supplementary material is said to contain the full codebase, but I did not see a URL."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: STAMPlus is a genuine extension of STAMP, not a rebrand. The structured JSON target list with explicit IDs, dynamic binding to a shared 200-class mask head, and one unified checkpoint across referring, open-vocabulary, instance, and remote-sensing segmentation are real contributions, and the benchmark sweep is broad with current baselines. The paper is also honest about its main soft spot: Table VIII(a) shows cIoU jumping from 77.4 to 89.1 when human boxes replace auto-generated Phase 1 cues, a larger gap than the margin over the previous SOTA. The authors frame this as an analysis and suggest RL or tool assistance, which is fair, but the multi-target results are reported only end-to-end, and the paper does not report how often Phase 1 drops, duplicates, or misorders a target. Since a lost ID is unrecoverable in Phase 2, the \"resolves the trilemma beyond single-target\" claim is better read as \"resolves it when the generative list is reliable.\" That is a limitation, not a fatal flaw.\n\nWhat the paper does well: the latency experiment is properly controlled (mean and std, 1-12 categories, single Phase 2 shared pass), the resolution scaling study isolates the effect of the 2560-3200 token budget, and the look-twice reasoning analysis is a sensible preliminary probe. The Phase 1->Phase 2 dependency is disclosed and even leveraged as a modular improvement path.\n\nThe soft spots are mostly reproducibility: no commit hash, no error bars on accuracy tables, missing training mixture ratios and hyperparameters, and a citation mismatch (M2SA in Table I cites ref [15], which is the MMR benchmark paper). These are fixable in revision. The method itself is coherent and the system-level claims are supported. I'd send this to peer review, and I'd be inclined to accept after moderate revision, assuming the code is actually released and Phase 1 parse-success rates are added. Worth a reading-group slot too -- it's a good case study in what \"structured prediction\" should mean for MLLM segmentation.","headline":"STAMPlus is a genuine extension of STAMP with a broad benchmark sweep; the Phase 1 target-list bottleneck is real but disclosed, and the paper deserves peer review.","tokens_in":27651,"tokens_out":3069,"would_cite":true,"duration_ms":27567,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims STAMPlus resolves the segmentation trilemma for multi-target segmentation with a single structured all-mask pass.","keywords":["multimodal large language models","image segmentation","structured all-mask prediction","non-autoregressive mask prediction","referring segmentation","open-vocabulary semantic segmentation","instance segmentation","remote sensing small-target segmentation"],"falsifier":"Run STAMPlus on an image with two same-category instances, then edit the Phase 1 output so the two IDs' entries are swapped in the JSON list (or drop the last entry) while keeping the image fixed. If the predicted masks do not swap identities accordingly, or if the dropped target's mask persists, the claimed dynamic ID binding through cached KV states is not what drives Phase 2; the paper's own oracle-box result predicts that such a perturbation should substantially degrade cIoU.","tokens_in":26570,"feed_emoji":"🎯","tokens_out":7029,"duration_ms":60076,"temperature":0.7,"pith_summary":"Multimodal large-language models that segment images face a three-way trade-off: accurate masks, natural dialogue, and fast inference. The paper claims that STAMPlus breaks this trilemma for multi-target segmentation by splitting the task into two phases: the model first writes a structured list of targets, each with an ID and an optional bounding box, then classifies every image patch against those IDs in a single parallel forward pass. Because mask prediction stays inside the token interface and does not use an external decoder, general dialogue ability is preserved, and because all masks are predicted at once, latency no longer grows with the number of targets. The paper reports state-of-the-art segmentation across referring, reasoning, open-vocabulary semantic, instance-aware, and remote-sensing small-target benchmarks using one unified checkpoint.","feed_headline":"All target masks in one pass: 13.5s drops to 5.16s","feed_subtitle":"A structured target list binds each ID to a mask, settling the speed-accuracy-dialogue trilemma for multi-target segmentation.","key_machinery":"The load-bearing mechanism is the two-phase all-mask pipeline with dynamic ID binding. Phase 1 is ordinary autoregressive text generation that produces a structured target list and a <SEG> trigger; Phase 2 prefills one mask token per image patch, fuses each token with its patch features and positional encoding, runs hybrid attention (causal on history, bidirectional among mask tokens), and classifies all tokens simultaneously. STAMPlus changes the prediction head from binary to a multi-class head with fixed capacity 200 and makes the label space instruction-defined by the Phase 1 list. KV-cache reuse is what makes Phase 2 cheap and what lets the mask tokens condition on the generated IDs.","core_discovery":"Structured All-Mask Prediction generalizes the authors' binary STAMP design to multi-target outputs. In Phase 1 the model autoregressively emits a JSON-style structured target list in which each entry assigns a numeric cls ID to a semantic category or object instance and may include a bounding box; the list ends with an in-vocabulary <SEG> trigger. In Phase 2, image-aligned [MASK] tokens, one per patch, are prefilled and processed in one non-autoregressive forward pass; a 200-way mask head labels each token as background or one of the active IDs, and predictions outside the Phase 1 list are discarded. Because Phase 2 reuses the Phase 1 KV cache, the mask tokens attend to the generated list, binding IDs to masks. The paper claims this resolves the trilemma beyond single targets: state-of-the-art cIoU and mIoU across referring, reasoning, open-vocabulary semantic, instance-aware, and remote-sensing small-target benchmarks; preserved general multimodal dialogue; and a 12-category mask latency reduction from 13.50 seconds for repeated STAMP inference to 5.16 seconds.","pith_inferences":["The explicit, parseable Phase 1 list turns the mask head into a per-sample open lexicon; the same binding could extend to panoptic video or 3D scenes without altering Phase 2, a direction the paper shows only as a qualitative panoptic example.","The 77.4-to-89.1 oracle-box jump implies a cheap testable upgrade: couple Phase 1 generation with a detector or reinforcement-learning-trained box proposer and measure whether the cIoU gain transfers to other benchmarks.","The fixed 200-class head capacity is an unexamined ceiling; scenes with more than about 200 targets would need hierarchical IDs or repeated structured lists, and no experiment probes that boundary."],"forward_implications":["Multi-category and multi-instance segmentation share a single Phase 2 forward pass, so inference latency no longer scales with the number of targets.","A single unified checkpoint, without task-specific fine-tuning, covers referring, reasoning, open-vocabulary semantic, instance-aware, and remote-sensing small-target segmentation.","Jointly scaling input resolution and mask-token budget from 1024-1280 to 2560-3200 tokens improves small-target gIoU by 5.5 points on RRSIS-D and 4.9 points on EarthReason.","Mixed training with visual-instruction data keeps general multimodal scores close to the backbone while adding segmentation, avoiding the dialogue collapse seen in embedding-prediction training.","The quality of the Phase 1 target specification gates Phase 2 accuracy: providing the target box raises RefCOCO-family average cIoU from 77.4 to 89.1."],"supporting_citations":[{"why":"The authors' prior CVPR paper that defines binary all-mask prediction and the segmentation trilemma, which STAMPlus extends.","marker":"[1]"},{"why":"The next-token patch-classification baseline, also the source of the RefCOCO-family training protocol used for STAMP.","marker":"[11]"},{"why":"The embedding-prediction paradigm whose pixel-level supervision motivates the dialogue-preservation requirement in the trilemma.","marker":"[12]"},{"why":"The strongest autoregressive patch-classification baseline that STAMPlus must beat, especially on RefCOCO and MUSE.","marker":"[20]"},{"why":"The base vision-language model whose general multimodal ability the method is designed to preserve.","marker":"[26]"},{"why":"The remote-sensing referring-segmentation benchmark and protocol used to measure small-target accuracy.","marker":"[51]"},{"why":"The geospatial reasoning benchmark and its strongest fine-tuned baseline, used for EarthReason evaluation.","marker":"[52]"},{"why":"The dense semantic supervision source used for open-vocabulary semantic segmentation training.","marker":"[53]"},{"why":"The panoptic mask source used for category- and instance-aware supervision in the unified training mixture.","marker":"[54]"}],"fun_headline_variants":["All target masks in one pass: 13.5s to 5.16s","One non-autoregressive pass for all masks, 2.6x faster","STAMPlus: structured list binds IDs to masks in one shot","Multi-target segmentation: one pass, all masks, 5.16s"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The trilemma claim collapses if Phase 1's generated target list is not a reliable, correctly ordered, complete specification of the targets; the paper's own experiment shows that replacing the auto-generated boxes with ground-truth boxes lifts RefCOCO-family cIoU from 77.4 to 89.1, so the Phase 1 list, not the mask classifier, is the current bottleneck.","fun_headline_variants_meta":{"raw":{"variants":["All target masks in one pass: 13.5s to 5.16s","One non-autoregressive pass for all masks, 2.6x faster","STAMPlus: structured list binds IDs to masks in one shot","Multi-target segmentation: one pass, all masks, 5.16s"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000432,"raw_usage":{"total_tokens":2284,"prompt_tokens":1104,"completion_tokens":1180,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":720,"completion_tokens_details":{"reasoning_tokens":1095}},"tokens_in":720,"tokens_out":1180,"duration_ms":8729,"temperature":1.0,"reasoning_tokens":1095,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:59:25.779338+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run STAMPlus on an image with two same-category instances, then edit the Phase 1 output so the two IDs' entries are swapped in the JSON list (or drop the last entry) while keeping the image fixed. If the predicted masks do not swap identities accordingly, or if the dropped target's mask persists, the claimed dynamic ID binding through cached KV states is not what drives Phase 2; the paper's own oracle-box result predicts that such a perturbation should substantially degrade cIoU.","supporting_citations":[{"cited_title":"Better, stronger, faster: Tackling the trilemma in mllm-based segmentation with simultaneous textual mask prediction,","cited_arxiv_id":null,"evidence_quote":"The authors' prior CVPR paper that defines binary all-mask prediction and the segmentation trilemma, which STAMPlus extends."},{"cited_title":"Text4Seg: Reimagining image segmentation as text generation,","cited_arxiv_id":null,"evidence_quote":"The next-token patch-classification baseline, also the source of the RefCOCO-family training protocol used for STAMP."},{"cited_title":"LISA: Reasoning segmentation via large language model,","cited_arxiv_id":null,"evidence_quote":"The embedding-prediction paradigm whose pixel-level supervision motivates the dialogue-preservation requirement in the trilemma."},{"cited_title":"Text4Seg++: Advancing image segmentation via generative language modeling,","cited_arxiv_id":null,"evidence_quote":"The strongest autoregressive patch-classification baseline that STAMPlus must beat, especially on RefCOCO and MUSE."},{"cited_title":"Rotated multi-scale interaction network for referring remote sensing image segmentation,","cited_arxiv_id":null,"evidence_quote":"The remote-sensing referring-segmentation benchmark and protocol used to measure small-target accuracy."},{"cited_title":"COCO-Stuff: Thing and stuff classes in context,","cited_arxiv_id":null,"evidence_quote":"The dense semantic supervision source used for open-vocabulary semantic segmentation training."}],"review_version":2}