{"id":"14b4efa4-31b1-461d-922b-f5ba09bf3113","arxiv_id":"2412.11890","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"SegMAN combines local neighborhood attention and Mamba state space scanning in a hybrid encoder-decoder to achieve strong semantic segmentation results on ADE20K, Cityscapes, and COCO-Stuff.","lead":"SegMAN is a new neural network for semantic segmentation that pairs neighborhood attention with state space scanning to capture both fine details and global context. The authors report accuracy gains on common segmentation benchmarks at comparable or lower compute than several efficient baselines.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SegMAN's SOTA and efficiency claims hinge on cross-paper baseline numbers that are not measured in a single harness; the 0.6–1.6 mIoU gaps and GFLOP/FPS advantages need controlled re-benchmarking.","rationale":"SegMAN's architecture is plausible and internally ablated; the LASS and MMSCopE components each show positive contributions, and the code is released. Those facts give some confidence that the model genuinely works. The unresolved point is whether it is actually better than the cited baselines in a way that supports the unqualified SOTA claim. The reported margins are small enough that training-protocol differences could erase them, and the efficiency measurements are not apples-to-apples. This is the load-bearing assumption because the paper's contribution is framed as 'new state-of-the-art while maintaining competitive computational efficiency'—if the baselines are re-run and the margins shrink, the central claim reduces to 'another efficient hybrid architecture with competitive accuracy,' which is a weaker but still acceptable contribution. I agree with the reader's weakest-assumption analysis, and the conditional verdict is the right response. The linear-time wording is also inaccurate given Stage-4 global attention, but correcting it would not change the numerical comparisons; it is a documentation issue rather than the key threat to the headline result.","tokens_in":19280,"tokens_out":9189,"duration_ms":84752,"concrete_test":"Run all rows of Tables 2 and 4 in a single controlled harness: use official checkpoints and code for SegNeXt, SegFormer, VWFormer, EDAFormer, and SegMAN; compute FLOPs with the same fvcore script at 512×512 (ADE20K/COCO-Stuff) and 2048×1024 (Cityscapes); measure FPS on the same L40S GPU with batch size 2 over 128 steps; and train SegMAN-S plus the two closest baselines (SegNeXt-L and EDAFormer-B) with three seeds under an identical augmentation, schedule, and crop-size protocol. If the gaps in mIoU fall below roughly 1 point or the GFLOP/FPS ordering changes, the headline SOTA/efficiency claim should be restricted to 'competitive' or to the specific measured configuration.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—SegMAN-B reaching 52.6 mIoU on ADE20K, 83.8 on Cityscapes, and 48.4 on COCO-Stuff with lower GFLOPs than SegNeXt-L, SegFormer-B3, and VWFormer-B3—treats numbers from different papers as if they were produced by one controlled comparison. SegMAN's FLOPs are computed in-house with fvcore, and its FPS are measured on an L40S GPU at batch size 2, while the baseline FLOPs and FPS are taken from papers that used different frameworks, GPUs, counting conventions, and training schedules. The accuracy deltas are single runs of 0.6–1.6 mIoU, a range that can easily be affected by random seed, crop sizes, augmentation, iteration count, and learning-rate schedule; the paper reports a 160K-iteration protocol with a stated 1e-6 learning rate that does not match the SegFormer protocol it says it follows, which further undermines reproducibility. The efficiency advantage is the second half of the headline claim, and Table 4 shows SegMAN-B having roughly half SegFormer-B3's GFLOPs but only 7% higher FPS, so the performance story is sensitive to which metric is used. Without re-running the closest baselines under identical conditions, the 'state-of-the-art at reduced cost' conclusion is not fully supported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"SegMAN proposes a semantic segmentation architecture with a hybrid encoder, SegMAN Encoder, that combines Neighborhood Attention (Natten) and a 2D selective-scan state-space model (SS2D) in its LASS token mixer, and a decoder whose MMSCopE module extracts multi-scale context using SS2D scans over concatenated feature maps at different resolutions. The paper reports ImageNet-1k classification results for the encoder and semantic segmentation results on ADE20K, Cityscapes, and COCO-Stuff-164K, claiming state-of-the-art accuracy at reduced FLOPs, with SegMAN-B reaching 52.6% mIoU on ADE20K, 83.8% on Cityscapes, and 48.4% on COCO-Stuff. The paper also provides systematic ablation studies of the LASS token mixer and MMSCopE decoder components, plus generalization experiments with SegNeXt and CGRSeg, and the code is promised publicly.","tokens_in":19570,"tokens_out":5622,"duration_ms":51673,"significance":"If the reported results are reproducible, the architecture is a useful contribution: it provides a concrete recipe for combining local attention with state-space models in dense prediction, and the decoder design respects linear-complexity scanning while preserving fine spatial detail through pixel-unshuffle. The ablation study is unusually thorough for this area and includes alternative token mixers, alternative fusion designs, and component-by-component deletions. The public code release is a further strength. However, the paper's headline claims are currently supported only by cross-paper baseline numbers and a strict 'linear-time' statement that conflicts with the use of global self-attention in Stage 4; the correctness of those claims is therefore not yet established at the level required for a journal.","major_comments":[{"comment":"The paper calls SegMAN a 'linear-time model' in the Abstract and repeatedly stresses linear complexity, but §3.2 states that SS2D is replaced with global self-attention in Stage 4 because the H/32 × W/32 resolution makes global attention feasible. Global self-attention has quadratic complexity in its token count, so the full model is not linear-time. Please either remove the global-attention stage from the definition of SegMAN, report the exact stage-4 complexity together with a 'near-linear' claim, or keep SS2D in Stage 4 for the main results. This is load-bearing because the efficiency claim is one of the paper's two headline contributions.","section":"Abstract and §3.2"},{"comment":"The training protocol states an AdamW learning rate of 1e-6 with 1500 warmup iterations for all segmentation runs, while saying that the SegFormer protocol is followed. SegFormer uses an initial learning rate of 6e-5 with poly decay; 1e-6 is 60× smaller and, as written, the reported accuracies are difficult to reconcile with such a schedule. Please clarify the exact learning-rate schedule (initial value, decay type, final value, and warmup length) and confirm whether '1e-6' is a typo.","section":"§4.1"},{"comment":"The headline efficiency comparisons mix measurement conventions: SegMAN FLOPs are computed with fvcore and SegMAN FPS are measured on an L40S GPU at batch size 2, while baseline FLOPs and FPS appear to be taken from their original papers. Table 4 illustrates the problem: SegMAN-B has roughly half the GFLOPs of SegFormer-B3 but only about 7% higher FPS, so the efficiency advantage is strongly metric-dependent. Please either re-run the closest baselines in the same harness or explicitly restrict the efficiency claim to FLOPs and mark which FPS numbers are not directly comparable.","section":"Tables 2 and 4, §4.2"},{"comment":"The 'state-of-the-art' claim is not fully supported by the selected comparison set. The table focuses on recent efficient segmentation models and includes Mask2Former only at its 47.7 mIoU configuration, omitting stronger mask-transformer baselines and other recently published results on ADE20K, Cityscapes, and COCO-Stuff. In addition, the table contains an inconsistent row labeled 'CGRSeg-T†' with 35.7M parameters in the small-model block, which does not match the CGRSeg-T configuration in the tiny block or Table 4. Please broaden the comparison to the current best published numbers and fix the labeling.","section":"Table 2 and §4.2"},{"comment":"All reported results appear to be single runs without error bars or multiple-seed statistics. Many of the claimed improvements are 0.6–1.6 mIoU, which can be within run-to-run variation for this training setup. Please provide variance estimates, specify the random seed protocol, or at minimum temper the claims and state the limitation explicitly.","section":"Tables 2, 3, and 4"}],"minor_comments":[{"comment":"The caption says 'three SegMAN Encoder variants' but the table lists four variants (Tiny, Small, Base, Large); please adjust the caption.","section":"Table 1"},{"comment":"The row 'CGRSeg-T†' with 35.7M parameters in the SegMAN-S block appears to be CGRSeg-B; please align the naming with Table 4.","section":"Table 2"},{"comment":"The baseline mIoU in Table 9 is 50.0, whereas the corresponding SegMAN-S row in main-paper Table 7 is 51.3; please explain this discrepancy or reconcile the two tables.","section":"Supplementary Table 9"},{"comment":"It is not stated whether the baseline FLOPs in Tables 2–4 were recomputed with fvcore in the same way as SegMAN or taken verbatim from the original papers; please state this explicitly.","section":"§4.1"},{"comment":"The FPS measurement protocol (GPU, batch size, number of warmup steps) is described only for SegMAN; please indicate which baseline FPS numbers were measured under the same conditions and which are cited from the literature.","section":"§4.2"},{"comment":"The module is spelled 'MMSCoPE' in §3.1 and 'MMSCopE' elsewhere; please unify the spelling.","section":"§3.1"},{"comment":"The 'Independent scans' ablation gains +0.2 mIoU but drops FPS from 128 to 65; this trade-off deserves a sentence in the main text rather than only appearing in the table.","section":"Table 7"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid empirical study with a useful architecture and unusually detailed ablations, but its central 'state-of-the-art at reduced cost' and 'linear-time' claims currently rest on heterogeneous baseline measurements and a training-protocol detail (1e-6 learning rate) that looks erroneous. I would like to see the authors either run controlled re-benchmarking of the closest baselines or substantially soften the efficiency and SOTA claims, and to clarify the Stage-4 complexity statement. With those changes the manuscript could be acceptable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: SegMAN is a competent engineering paper that gets real gains by stacking neighborhood attention and SS2D, plus a decoder that scans multi-scale features in one pass. The experiments are extensive and the ablations are honest. But the headline claims outrun the evidence: 'linear-time' is not true with global self-attention in stage 4, and the SOTA comparisons rely on cross-paper FLOPs/FPS numbers that are not measured in one harness. I'd send it to review, but ask for a re-scoped title/abstract and a controlled re-benchmark of closest baselines before acceptance.\n\nWhat is new and good: the LASS token mixer is a straightforward but effective combination of two existing modules, and the MMSCopE decoder's pixel-unshuffle trick to run one SS2D scan over three scales is genuinely neat. The ablations show both components earn their keep: removing SS2D costs 3.9 mIoU, removing Natten costs 1.5. The generalization experiments swapping encoder/decoder into SegNeXt and CGRSeg are the kind of evidence I like to see. Code is public. This is a useful, reproducible engineering contribution.\n\nThe soft spots are mostly in the claims, not the method. The paper says 'linear-time' repeatedly, but Stage 4 blocks replace SS2D with global self-attention. At 1/32 resolution that is not the dominant cost, but it means the complexity statement is false as written. The SOTA claim is also scoped too broadly. Table 2 includes Mask2Former at 47.7 on ADE20K, which is not comparable to SegMAN-B at 52.6 because the training recipes differ; SegMAN's real comparison is against efficient single-scale encoders like SegFormer, SegNeXt, and VWFormer. In that regime the gains look solid, but the abstract should say so.\n\nMore worrying is the efficiency comparison. SegMAN's FLOPs are computed with fvcore and its FPS on an L40S, while baseline numbers are pulled from papers that used different hardware, frameworks, and counting conventions. The accuracy deltas are 0.6 to 1.6 mIoU, single runs with no error bars. That is enough to make the headline margin real but not enough to pin down. Also, the implementation details say the LR is 1e-6 with 1500 warmup; that does not match the SegFormer protocol they cite and it looks like a typo—worth verifying, because it affects reproducibility. No commit hash or environment details, though the code is public.\n\nNone of this sinks the paper. The architecture is plausible, the ablations are systematic, and the gains are consistent across three datasets and three model sizes. I'd send it to a serious referee. The revisions I'd want are: restrict the SOTA claim to the efficient regime, fix the linear-time wording, add variance or at least a seed comparison, and re-run the two or three closest baselines in the same harness to confirm the FLOPs/FPS story.","headline":"Useful hybrid architecture with honest ablations; headline claims about linear-time and SOTA need re-scoping.","tokens_in":20080,"tokens_out":2253,"would_cite":true,"duration_ms":19216,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SegMAN claims that mixing a state-space global scan with sliding-window local attention gives state-of-the-art semantic segmentation at linear time and lower cost across ADE20K, Cityscapes, and COCO-Stuff.","keywords":["semantic segmentation","state space models","Mamba","neighborhood attention","multi-scale context","linear-time architecture","ADE20K","Cityscapes"],"falsifier":"Re-run the main baselines (SegNeXt-L, SegFormer-B3, VWFormer-B3, EDAFormer-B) with the paper's own training settings, resolutions, and hardware, then remeasure mIoU, GFLOPs, and FPS; if SegMAN-B no longer beats them by roughly the reported margins (for example, 1.6 mIoU on ADE20K and 0.6 on Cityscapes), the central comparison claim would be falsified.","tokens_in":19040,"feed_emoji":"🧩","tokens_out":7454,"duration_ms":62007,"temperature":0.7,"pith_summary":"This paper tries to establish that a single network can hold global context, local detail, and multi-scale representation at the same time, and that this combination is what pushes semantic segmentation accuracy forward. The proposed SegMAN fuses a two-dimensional selective-scan state space model with sliding-window neighborhood attention in a token mixer called LASS, and adds a decoder module, MMSCopE, that scans several scales in one pass. If the paper is right, high-resolution segmentation no longer needs quadratic self-attention or fixed predefined windows: SegMAN is linear-time, adapts to changing input resolutions, and reports higher mIoU than larger prior models on ADE20K, Cityscapes, and COCO-Stuff-164K.","feed_headline":"State-space plus local attention tops segmentation benchmarks","feed_subtitle":"SegMAN-B scores 52.6 mIoU on ADE20K and 83.8 on Cityscapes with fewer FLOPs than prior leaders.","key_machinery":"The load-bearing object is the LASS token mixer: a serial stack of Neighborhood Attention (a sliding-window attention that keeps translational equivalence and linear complexity) with the two-dimensional selective-scan block SS2D (a Mamba-style state space model that scans each feature map in four directions for global context in linear time), plus a residual connection around SS2D and a 1x1 convolution that fuses local and global streams. The second mechanism is MMSCopE, the decoder module: it creates coarser feature maps with strided convolutions, losslessly downsamples them with pixel unshuffle to a common resolution, concatenates them along channels, and extracts multi-scale context with a single SS2D scan, followed by a 1x1 convolution and upsampling. Together they let the network cover the full feature map at every resolution while preserving fine detail.","core_discovery":"The central claim is that omni-scale context modeling — global dependencies, local boundary detail, and multiple intermediate scales at once — can be delivered in linear time by a hybrid encoder-decoder. In the encoder, each LASS block serially stacks Neighborhood Attention (sliding-window local attention) and the VMamba SS2D block (a four-directional selective scan), with a residual shortcut around the SS2D; the last stage uses global attention. In the decoder, MMSCopE downsamples the fused feature map with strided convolutions, uses pixel unshuffle to bring the scales to one resolution without information loss, and runs a single SS2D scan over the channel-concatenated result to extract multi-scale context. The paper reports that the Base model reaches 52.6% mIoU on ADE20K, 83.8% on Cityscapes, and 48.4% on COCO-Stuff-164K, while the Base encoder reaches 85.1% ImageNet-1k top-1 accuracy.","pith_inferences":["The paper's motivation implies a concrete test: the accuracy gap between SegMAN and fixed-window multi-scale models should grow with input resolution, since fixed windows lose full-map coverage.","The single-scan channel-concatenation trick is an efficiency choice; comparing it to independent per-scale scans at higher resolutions or on non-square inputs would separate architectural benefit from GPU convenience.","The local-plus-global recipe points to a general design pattern for high-resolution dense prediction, so extending the same hybrid mixer to video or 3D segmentation is a natural test of its scope.","Because the headline comparisons inherit published baseline numbers, the internal ablations (removing SS2D costs 3.9 mIoU; removing Neighborhood Attention costs 1.5 mIoU) are the part of the evidence that does not depend on external measurement conventions."],"forward_implications":["Quadratic global self-attention is not required for top segmentation accuracy; a four-direction state-space scan plus local attention matches or beats attention-based models at lower FLOPs.","Fixed-window multi-scale attention loses its advantage as input resolution grows, because a single scan over full multi-scale maps keeps complete feature-map coverage at any resolution.","SegMAN's encoder and decoder transfer independently: replacing a base model's encoder or decoder with either component raises mIoU while reducing parameters and FLOPs.","The encoder generalizes beyond semantic segmentation, improving panoptic and instance segmentation when used as a backbone in Mask DINO."],"supporting_citations":[{"why":"Supplies the dynamic selective state-space model that gives global context linear-time scanning.","marker":"[16]"},{"why":"Supplies the SS2D four-directional scan block used in LASS and MMSCopE, and serves as a backbone baseline.","marker":"[28]"},{"why":"Supplies Neighborhood Attention, the sliding-window local operator credited with fine detail retention.","marker":"[19]"},{"why":"Supplies the SegFormer training protocol and MLP decoder baseline that SegMAN's numbers are compared against.","marker":"[56]"},{"why":"Supplies SegNeXt and MSCAN, the strongest segmentation comparison and the source of one swapped decoder and encoder.","marker":"[18]"},{"why":"Supplies VWFormer, the predefined-window multi-scale baseline whose fixed scales motivate SegMAN's adaptive design.","marker":"[57]"},{"why":"Supplies EDAFormer, the embedding-free attention baseline whose downsampling motivates preserving fine details.","marker":"[60]"},{"why":"Supplies pixel unshuffle, the lossless downsampling that lets MMSCopE concatenate scales for one SS2D scan.","marker":"[42]"},{"why":"Supplies the ADE20K benchmark and its 150-class evaluation used for ablations and main comparisons.","marker":"[65]"}],"fun_headline_variants":["SegMAN: Omni-scale context in linear time","State-space + local attention for omni-scale segmentation","Linear-time hybrid model tops ADE20K and Cityscapes","Efficient omni-scale modeling with SSM and attention","SegMAN beats prior leaders with lower GFLOPs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison with prior models assumes that the published FLOPs, FPS, and mIoU numbers were measured under equivalent training recipes and hardware; if baseline numbers are not directly comparable, the reported accuracy and efficiency gains could be smaller.","fun_headline_variants_meta":{"raw":{"variants":["SegMAN: Omni-scale context in linear time","State-space + local attention for omni-scale segmentation","Linear-time hybrid model tops ADE20K and Cityscapes","Efficient omni-scale modeling with SSM and attention","SegMAN beats prior leaders with lower GFLOPs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000722,"raw_usage":{"total_tokens":3294,"prompt_tokens":1055,"completion_tokens":2239,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":671,"completion_tokens_details":{"reasoning_tokens":2159}},"tokens_in":671,"tokens_out":2239,"duration_ms":15256,"temperature":1.0,"reasoning_tokens":2159,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:28:00.224061+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the main baselines (SegNeXt-L, SegFormer-B3, VWFormer-B3, EDAFormer-B) with the paper's own training settings, resolutions, and hardware, then remeasure mIoU, GFLOPs, and FPS; if SegMAN-B no longer beats them by roughly the reported margins (for example, 1.6 mIoU on ADE20K and 0.6 on Cityscapes), the central comparison claim would be falsified.","supporting_citations":[{"cited_title":"Neighborhood attention transformer","cited_arxiv_id":null,"evidence_quote":"Supplies Neighborhood Attention, the sliding-window local operator credited with fine detail retention."},{"cited_title":"Segformer: Simple and efficient design for semantic segmentation with transform- ers","cited_arxiv_id":null,"evidence_quote":"Supplies the SegFormer training protocol and MLP decoder baseline that SegMAN's numbers are compared against."},{"cited_title":"Segnext: Rethink- ing convolutional attention design for semantic segmenta- tion","cited_arxiv_id":null,"evidence_quote":"Supplies SegNeXt and MSCAN, the strongest segmentation comparison and the source of one swapped decoder and encoder."},{"cited_title":"Multi-scale rep- resentations by varying window attention for semantic seg- mentation","cited_arxiv_id":null,"evidence_quote":"Supplies VWFormer, the predefined-window multi-scale baseline whose fixed scales motivate SegMAN's adaptive design."},{"cited_title":"Embedding-free transformer with inference spatial reduction for efficient se- mantic segmentation","cited_arxiv_id":null,"evidence_quote":"Supplies EDAFormer, the embedding-free attention baseline whose downsampling motivates preserving fine details."},{"cited_title":"Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network","cited_arxiv_id":null,"evidence_quote":"Supplies pixel unshuffle, the lossless downsampling that lets MMSCopE concatenate scales for one SS2D scan."}],"review_version":1}