{"id":"ba8aab1d-49c0-4bda-b99f-15d7b93fc6c5","arxiv_id":"2509.06321","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A single MLLM can segment images by generating box coordinates plus compact 'semantic brick' mask tokens, matching or beating many decoder-based methods without task-specific fine-tuning.","lead":"This paper turns image segmentation into a text-generation task: an MLLM outputs words and special tokens that describe object masks, with no extra decoder. It reports strong results on many segmentation benchmarks and shows that one general model can handle natural and remote sensing images.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 64×64 box-wise quantization may cap mask fidelity for small/thin remote-sensing objects; Fig. 13's resolution study never tests this regime, so the representation ceiling is not settled.","rationale":"The reader's weakest assumption is also the point I find most load-bearing: the adequacy of the 64×64 box-wise representation. My reading of Section III-C and Fig. 7 is that the 126 brick tokens are run-length tokens and arbitrary run lengths can be composed, so I do not press the 'fixed vocabulary cannot express runs' version of the concern. The real risk is the 64-bin spatial quantization, which is precisely the untested ceiling for small and thin objects. The paper deserves credit for the resolution study in Section V-E.1, the no-refiner results in Tables II and V, and the explicit disclosure in Table VII that it underperforms SegEarth-R1; those disclosures make the conclusion's blanket 'consistently surpasses state-of-the-art' internally inconsistent. But the more technical issue is that the representation's ceiling on aerial imagery has not been separated from model capability. An oracle reconstruction test is cheap and would settle it. If the oracle ceiling is high, the concern does not land and the unified text-as-mask paradigm is strengthened; if it is low, the central claim needs to be scoped to benchmarks where the 64×64 grid is adequate. This is a scoping issue, not a refutation, so it does not change the conditional verdict.","tokens_in":28853,"tokens_out":11732,"duration_ms":114460,"concrete_test":"Run an oracle reconstruction test on RRSIS-D val and on a thin-structure subset of RefCOCO: take every ground-truth mask, quantize it exactly as in Section III-C (64×64 mask grid, 64 coordinate bins, run lengths encoded with the 126 bricks), decode the B-SD back to pixels, and compute gIoU, cIoU, and Boundary IoU against the original masks. If oracle reconstruction already falls below the 74.4 SegEarth-R1 score, the representation ceiling explains the Table VII gap and the 'surpasses SOTA' claim must be qualified. As a complementary check, fine-tune or evaluate Text4Seg++ with 128×128 B-SD on RRSIS-D; a large gain on boundary metrics would confirm that the default 64×64 grid is the limiting factor.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim depends on the premise that the B-SD format of Section III-C is an adequate representation for every target mask. Section III-C.2 fixes 64 discrete coordinate bins for boxes and a 64×64 grid for mask content, with 126 semantic-brick tokens encoding run lengths 1–63. Run lengths can be chained, so the fixed brick vocabulary is not by itself the problem; the real ceiling is spatial quantization: mask detail is bounded by roughly 1/64 of the image width and box localization by the same 64-bin grid. For RefCOCO-style objects, which typically span many grid cells, this is acceptable, and the resolution ablation in Section V-E.1 and Fig. 13 shows monotonic gains from 32×32 to 80×80. However, that ablation is run only on RefCOCO-family benchmarks. On RRSIS-D, an 800×800 aerial benchmark, many referents are only a few pixels wide and can fall below one 64-bin grid cell. The paper's own Table VII reports Text4Seg++ at 70.8 average versus 74.4 for SegEarth-R1, and no boundary-sensitive metric is reported anywhere, so the hypothesis that the 64×64 representation ceiling, not the MLLM's learning or reasoning ability, causes the gap is untested. This is load-bearing because if the oracle quantized representation already loses several points on RRSIS-D, then the headline claim that Text4Seg++ 'consistently surpasses state-of-the-art' fails for that benchmark at the default resolution, regardless of how well the language model is trained.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Text4Seg++ (and its predecessor Text4Seg), a decoder-free paradigm that casts image segmentation as pure text generation by representing masks as semantic descriptors. The key innovations are image-wise semantic descriptors with row-wise run-length encoding (R-RLE) and, for Text4Seg++, box-wise semantic descriptors (B-SD) that couple a quantized bounding box with a compact 64x64 binary mask encoded through a vocabulary of 126 'semantic bricks'. A single MLLM (Qwen2-VL-7B or Qwen2.5-14B) is fine-tuned with LoRA on a mixed corpus of segmentation, grounding, and VQA data, then evaluated on ten benchmarks covering referring, generalized referring, reasoning, multi-object reasoning, open-vocabulary, remote-sensing, comprehension, and VQA tasks, all without task-specific fine-tuning or architectural modification. The paper reports strong results on RefCOCO-family referring segmentation, MUSE, EarthReason, and referring expression comprehension, alongside ablations on resolution, R-RLE, and semantic bricks.","tokens_in":29202,"tokens_out":4251,"duration_ms":35665,"significance":"If the central claim holds, this is a significant advance: it demonstrates that a decoder-scale MLLM can perform dense segmentation purely through autoregressive text generation, with no mask decoder and no architecture changes, while remaining competitive with or superior to specialized and generalist methods on several benchmarks. The paper includes extensive multi-benchmark evaluation, token-length analysis showing a 74% reduction from R-RLE and further compression from semantic bricks, and ablations that support the design choices. The method is also shown to be backbone-agnostic, working with Qwen2-VL, Deepseek-VL2, and InternVL3. However, the paper's own tables contradict the headline claim of 'consistently surpassing state-of-the-art': Seg-Zero beats Text4Seg++ on ReasonSeg, and both SegEarth-R1 and RMSIN beat it on RRSIS-D. The representation ceiling of the 64x64 B-SD grid for small remote-sensing objects is also not tested, leaving a load-bearing question unresolved.","major_comments":[{"comment":"The claim that Text4Seg++ 'consistently surpasses existing state-of-the-art methods' and 'achieves this superior performance without any task-specific fine-tuning' is contradicted by the paper's own results. In Table IV (ReasonSeg), Seg-Zero (56.2) outperforms Text4Seg++ (54.5) on average gIoU/cIoU. In Table VII (RRSIS-D), SegEarth-R1 (74.4) and RMSIN (72.4) both outperform Text4Seg++ (70.8). The conclusion should be qualified to name the benchmarks where Text4Seg++ actually sets the state of the art, and the authors should discuss the observed gaps on ReasonSeg and RRSIS-D. This is load-bearing because the abstract and conclusion frame the contribution as universal superiority, and the current wording is not supported by the evidence in the tables.","section":"Abstract; Section V-A (Conclusion)"},{"comment":"The B-SD representation fixes the mask grid at 64x64 and the bounding boxes at 64 discrete coordinate bins, so the representable mask fidelity is bounded by roughly one sixty-fourth of the image width. The resolution ablation in Figure 13 is conducted only on the RefCOCO-family benchmarks, where objects are typically large relative to the grid. On RRSIS-D, whose images are 800x800 pixels, many referents are only a few pixels wide and can fall below one 64-bin grid cell. The paper reports no boundary-sensitive metric anywhere, and it does not evaluate higher B-SD resolutions on RRSIS-D. Consequently, the hypothesis that the 64x64 representation ceiling, rather than the MLLM's learning or reasoning ability, explains the RRSIS-D gap (70.8 vs. 74.4 for SegEarth-R1) is untested. I recommend adding an oracle experiment that quantizes the ground-truth masks into the B-SD format and computes the resulting IoU ceiling, or running B-SD at higher resolution on RRSIS-D to separate representation error from model error.","section":"Section III-C.2; Section V-E.1; Table VII; Figure 13"}],"minor_comments":[{"comment":"The row 'Text4Seg++ Qwen2-7B ✓' appears in the 'Generalist Segmentation Models (≥13B)' block, but it is a 7B model and also appears in the 7B block; this appears to be a table formatting error.","section":"Table III"},{"comment":"The phrase 'consistently outperforms state-of-the-art models' is too strong given the exceptions in Tables IV and VII; consider rewording to 'outperforms state-of-the-art methods on most benchmarks, with notable exceptions on ReasonSeg and RRSIS-D'.","section":"Abstract; Section V-A"},{"comment":"Reference [11] is cited as 'Towards open vocabulary learning: A survey' in the reference list, but Table II attributes the SETOKIM method to [11]; the citation numbering appears to be misaligned.","section":"Section II-B; Reference [11]; Table II"},{"comment":"The caption lists 'I-SD, B-SD, and B-SD without semantic bricks' but the legend labels in the figure are not immediately mappable to these configurations; please make the legend explicit.","section":"Figure 7"},{"comment":"The RRSIS-D dataset description states 17,402 image-mask-expression triplets, while Table I reports 12k images and 61k samples; clarify the relationship between these counts (e.g., number of expressions versus number of unique images).","section":"Section IV-B; Table I"},{"comment":"Table X evaluates only Text4Seg on VQA and RES benchmarks, not Text4Seg++; if the compatibility-with-visual-understanding claim is intended for Text4Seg++ as well, please include its VQA results or state clearly that this evaluation covers only Text4Seg.","section":"Section IV-C.7; Table X"}],"recommendation":"major_revision","confidential_remarks":"The paper is technically solid in most respects: the ablations are informative, the token-efficiency analysis is convincing, and the method shows clear gains on RefCOCO, MUSE, and EarthReason. The main issue is that the abstract and conclusion overclaim 'consistent state-of-the-art' when two of the paper's own tables contradict this. The representation-ceiling concern is legitimate and should be addressed with an oracle quantization experiment or higher-resolution B-SD evaluation on RRSIS-D. The paper fits the scope of the journal and is likely publishable after the claims are calibrated and the missing experiment is added."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a real step forward for decoder-free MLLM segmentation, but the paper oversells itself.\n\nWhat you should know: Text4Seg++ is not a trivial extension of the earlier Text4Seg. The box-wise semantic descriptor — a quantized bounding box plus a run-length mask encoded as 'semantic bricks' — is a concrete, compact representation that genuinely recasts segmentation as next-token prediction. The unified LoRA fine-tuned model sets new SOTA on the RefCOCO family, MUSE, EarthReason, and referring expression comprehension, and it does so without mask decoders or task-specific heads. The ablations on resolution, R-RLE, and bricks are thorough and support the design choices.\n\nThe main soft spot is the abstract's claim that Text4Seg++ 'consistently surpasses state-of-the-art.' That is not true on the paper's own numbers: Seg-Zero beats it on ReasonSeg, SegEarth-R1 beats it on RRSIS-D, and specialized open-vocabulary models still dominate. The authors do acknowledge these gaps in the body, but the abstract and conclusion overstate. A revision should narrow the claim.\n\nThe stress-test concern about the 64x64 grid is worth taking seriously. The resolution ablation runs only on RefCOCO-family images, where objects are large relative to the grid. RRSIS-D is 800x800 aerial imagery, and one grid cell is 12.5 pixels; small referents can fall below that. The paper never reports a boundary-sensitive metric, and it never tests what an oracle-quantized mask achieves on RRSIS-D. So we cannot tell whether the gap is the representation ceiling or the MLLM's learning. That is a load-bearing question for the 'pure text segmentation' claim, but it is answerable with an oracle experiment.\n\nOther gaps are minor by comparison: no code, no comparison to HiMTok or ALTo (both cited), and the SAM-refined variant is used in some tables but not consistently labeled. The core approach is sound and the evaluation is extensive. I see no circularity: training uses ground-truth masks, evaluation is against held-out labels.\n\nThis paper deserves a serious referee. The encoding idea is novel and the empirical coverage is above average. I would recommend the editor send it to review with the request that the authors fix the overclaims, add the missing baselines, release code, and address the quantization ceiling. If those land, this is a solid journal contribution.","headline":"A genuinely new text-as-mask encoding with strong results on several benchmarks, but the 'consistently surpasses SOTA' claim overreaches and the representation ceiling on small remote-sensing objects is untested.","tokens_in":29793,"tokens_out":2790,"would_cite":true,"duration_ms":24068,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that image segmentation can be cast as pure text generation, replacing mask decoders with bounded boxes and brick codes that a multimodal large language model writes autoregressively, and that this approach beats…","keywords":["image segmentation","multimodal large language models","text-as-mask","semantic descriptors","semantic bricks","next-brick prediction","referring expression segmentation","reasoning segmentation"],"falsifier":"Take a dataset of masks with thin structures such as power lines, hair, or small boats in satellite imagery, render each ground-truth mask through the 64x64 brick encoding, and measure the IoU ceiling between the encoded mask and the original. If that ceiling is already below the reported gains, then the representation, not the model, bounds performance and the pure-text claim fails at the encoding stage.","tokens_in":28626,"feed_emoji":"🧱","tokens_out":6210,"duration_ms":55258,"temperature":0.7,"pith_summary":"This paper tries to establish that a multimodal large language model (MLLM) can perform image segmentation as pure text generation, replacing the usual mask decoder with a short structured string: a quantized bounding box, a label or region tag, and brick codes that draw the mask on a 64x64 grid. The claim is that this representation is not only simpler but also more compact, more precise, and more scalable than decoder-based alternatives. A single model trained once on mixed segmentation and visual-question data is said to transfer across referring, reasoning, open-vocabulary, and remote-sensing segmentation without task-specific fine-tuning. If true, segmentation becomes a capability that an instruction-tuned multimodal language model can acquire by changing its training data rather than its architecture.","feed_headline":"Masks as text: one model beats decoder-based segmenters","feed_subtitle":"A single MLLM writes boxes and brick codes, winning referring, reasoning, and remote-sensing benchmarks without task fine-tuning.","key_machinery":"The load-bearing object is the box-wise semantic descriptor (B-SD), a text tuple of the form <ref>label or region tag</ref><box>[[x1 y1 x2 y2]]</box><seg>[[brick codes]]</seg>. Inside the box, the mask is drawn with semantic bricks: 63 foreground and 63 background special tokens, each standing for a run of filled or empty cells on a 64x64 grid, so segmentation becomes next-brick prediction in raster-scan order. This compresses a mask to about 150 tokens on the referring segmentation benchmark, versus roughly 768 tokens for the earlier full-image patch labels, while providing finer spatial detail. The mechanism works because the autoregressive language model jointly decides where to look, what to call the object, and how to paint it, and the box constraint removes the background-token dominance that makes whole-image text masks verbose and less scalable.","core_discovery":"The central claim is that segmentation masks can be encoded entirely as text and predicted by a generative language model, with no auxiliary mask decoder and no architectural modification. The paper introduces box-wise semantic descriptors: for each object, the model outputs a referring label or abstract region tag, a bounding box quantized into 64 discrete bins, and a sequence of semantic bricks that reconstruct the binary mask on a 64x64 canvas in raster-scan order. The authors report that this next-brick prediction formulation outperforms prior decoder-based and polygon-based generalist segmenters on referring expression segmentation, generalized referring segmentation, multi-object reasoning segmentation, referring expression comprehension, and remote sensing benchmarks, while remaining compatible with several existing MLLM backbones. The earlier image-wise variant, which labels every image patch with text and applies row-wise run-length encoding, is presented as already competitive on open-vocabulary and visual-question tasks.","pith_inferences":["Because the mask is expressed with a fixed vocabulary of 126 bricks and a 64x64 grid, the method's ceiling is representational: finer masks or unusual shapes would require more bricks or a learned codebook, not just more model capacity.","The box-first output stream makes grounding and segmentation share one set of token probabilities, so a model that misplaces a box could in principle be corrected by decoding the brick sequence back into a mask and checking consistency.","The remote-sensing gains suggest a testable extension to other sparse-mask domains, such as medical or aerial imagery, where boxes are natural priors and small structures remain within the 64x64 grid.","Treating the brick sequence as a latent code opens an avenue for verification or chain-of-thought: if the model can regenerate the box from its own bricks, the representation is self-consistent enough for agentic loops."],"forward_implications":["A single model trained once on the mixed corpus sets state-of-the-art numbers on referring expression segmentation at both 7B and 13B scales, with reported RefCOCO val cIoU of 81.5 without any mask refiner.","The same model reaches a reported 91.1 average Acc@0.5 on referring expression comprehension, suggesting that dense mask supervision sharpens grounding even when only the bounding box is evaluated.","On multi-object reasoning segmentation, the paper reports an average score of 63.8, about 11 points above the prior best, without task-specific fine-tuning.","At 64x64 resolution, the brick-based mask matches or exceeds the quality of the same model with an external mask refiner, making the refiner optional rather than necessary.","Semantic bricks cut token length from 767.6 to 150.4 at 64x64 resolution, so higher resolution no longer implies proportionally longer output sequences."],"supporting_citations":[{"why":"The paper's earlier conference version that introduced image-wise semantic descriptors and row-wise run-length encoding, and the direct predecessor Text4Seg++ extends and compares against.","marker":"[25]"},{"why":"The decoder-based reasoning segmentation model that defines the embeddings-as-mask paradigm the paper argues against and evaluates against on reasoning and referring benchmarks.","marker":"[15]"},{"why":"The patch-based visual representation that motivates the image-wise semantic descriptor design.","marker":"[26]"},{"why":"The high-resolution vision-language backbone used for the main Text4Seg++ experiments.","marker":"[27]"},{"why":"The low-rank adaptation method used for all fine-tuning, which keeps the MLLM architecture unchanged.","marker":"[66]"},{"why":"The optional off-the-shelf mask refiner used as a post-processing step and as an ablation comparison for pixel-level mask quality.","marker":"[21]"},{"why":"The geospatial reasoning segmentation work whose benchmark and results provide the remote sensing comparison for domain generalization.","marker":"[48]"},{"why":"The benchmark defining generalized referring expression segmentation with multi-target and no-target cases, a core evaluated task.","marker":"[72]"},{"why":"The multi-object reasoning segmentation benchmark and baseline that Text4Seg++ outperforms by a large margin.","marker":"[19]"}],"fun_headline_variants":["Text-as-mask: language model writes segmentation codes","No decoder needed: MLLM outputs boxes and brick codes","Segmentation via next-brick prediction, outperforming decoders","Text4Seg++: boxes and semantic bricks for mask generation","Generative text model replaces segmentation decoders"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that every target mask is faithfully representable as a 64x64 grid of foreground and background brick runs inside a quantized bounding box; if masks need finer detail than that grid can express, the pure-text approach cannot represent them no matter how well the model learns.","fun_headline_variants_meta":{"raw":{"variants":["Text-as-mask: language model writes segmentation codes","No decoder needed: MLLM outputs boxes and brick codes","Segmentation via next-brick prediction, outperforming decoders","Text4Seg++: boxes and semantic bricks for mask generation","Generative text model replaces segmentation decoders"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000282,"raw_usage":{"total_tokens":1709,"prompt_tokens":1025,"completion_tokens":684,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":641,"completion_tokens_details":{"reasoning_tokens":604}},"tokens_in":641,"tokens_out":684,"duration_ms":5934,"temperature":1.0,"reasoning_tokens":604,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T16:17:49.241691+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a dataset of masks with thin structures such as power lines, hair, or small boats in satellite imagery, render each ground-truth mask through the 64x64 brick encoding, and measure the IoU ceiling between the encoded mask and the original. If that ceiling is already below the reported gains, then the representation, not the model, bounds performance and the pure-text claim fails at the encoding stage.","supporting_citations":[{"cited_title":"Text4seg: Reimagining image segmentation as text generation,","cited_arxiv_id":null,"evidence_quote":"The paper's earlier conference version that introduced image-wise semantic descriptors and row-wise run-length encoding, and the direct predecessor Text4Seg++ extends and compares against."},{"cited_title":"Segment anything,","cited_arxiv_id":null,"evidence_quote":"The optional off-the-shelf mask refiner used as a post-processing step and as an ablation comparison for pixel-level mask quality."}],"review_version":2}