{"id":"58bbe12e-05f4-43ac-8e0d-7cb979a1b224","arxiv_id":"2412.14546","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"S3-Mamba, a Mamba-based segmentation network with tensor-based multi-scale attention and regularized curriculum learning, reports higher small-lesion segmentation accuracy on three medical imaging datasets.","lead":"This paper proposes S3-Mamba, a Mamba-based neural network that aims to improve segmentation of small lesions in medical images by combining channel attention, multi-scale tensor attention, and a curriculum learning strategy. If the results replicate, it offers a practical tool for detecting small lesions earlier, though the evidence currently rests on limited public comparisons and a private dataset.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Curriculum learning in Eqs. (6)-(7) is formally ill-posed: v is not an optimization variable, no update rule for v is stated, and the rank-based term in g(v) is independent of v; the reported CL gains cannot be reproduced from the paper alone.","rationale":"The paper's central empirical claim is plausible and the architecture contributions are well-motivated. The single most load-bearing concern is the curriculum learning formalization, because it is one of the three named contributions and the ablations attribute concrete gains to it. The reader's specific degeneracy argument is partially a misreading: Eq. (6) minimizes over ϕ, not over v, so 'v_i = 0' does not follow directly. Nevertheless, the deeper issue stands: the objective is not well-defined without an explicit update rule for v, and g(v) contains a term independent of v. The released code can settle this immediately. If the code uses a different scheduler, the paper's equations are wrong; if it uses Eq. (6) as a joint minimization, the training would collapse to v=0. Either way the paper needs revision. This does not overturn the empirical comparison, so the verdict remains conditional.","tokens_in":15999,"tokens_out":12062,"duration_ms":109966,"concrete_test":"Inspect the released training code (github.com/ErinWang2023/S3-Mamba) to extract the exact sample-weight update. If v_i is updated by a heuristic based on loss rank and lesion size (rather than minimizing Eq. (6)), then the paper's equations misrepresent the method and must be corrected. If the code instead evaluates the gradient of Eq. (6) with respect to v, verify that v does not collapse to 0 (the only v-dependent term is positive and l_i ≥ 0, so joint minimization would drive v → 0); a nonzero learned v would prove the implemented objective differs from the stated one.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The regularized curriculum learning contribution is a central part of the 'training strategy' dimension, and the ablation (Tables 3-4) attributes measurable gains to +CL. Yet Eqs. (6)-(7) do not define a computable training algorithm. Eq. (6) reads min_ϕ Σ v_i l_i + g(v), with v ∈ [0,1]^N 'determined by the Difficulty Measurer'. Because v is not a free variable, the reader's specific concern that minimizing over v yields v_i=0 is not literally entailed by the min_ϕ notation. However, the equation is still ill-posed: g(v) in Eq. (7) contains λΣ 1/Frank(l_i), which has no v_i, and the only v-dependent term is (1−λ)Σ v_i^2. The text says the Difficulty Measurer 'initializes sample weights based on lesion size' and 'dynamically updates them according to loss values', but no update rule, gradient, or pseudo-code is provided. Thus the objective's dependence on ϕ through v is unspecified, and the 'regularized' mechanism cannot be re-implemented from the paper. Since the ablation shows CL contributes to the reported mIoU gains, this unstated rule is load-bearing for the paper's claim of a novel curriculum learning strategy.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes S3-Mamba, a Mamba-based U-Net for segmenting small lesions, with three main contributions: an Enhanced Visual State Space (EnVSS) block that adds channel-wise attention and residual connections to a VSS block; a Tensor-based Cross-feature Multi-scale Attention (TCMA) that fuses input, edge, and intermediate prediction features at multiple patch scales; and a regularized curriculum learning strategy with a Difficulty Measurer and Training Scheduler. The model is evaluated on ISIC2018, CVC-ClinicDB, and a private Lymph dataset, reporting large improvements in small-lesion mIoU and DSC over nine baselines, with ablation studies showing positive contributions from each component.","tokens_in":16277,"tokens_out":7998,"duration_ms":65247,"significance":"The core idea of addressing small-lesion segmentation through channel, spatial, and training-level mechanisms is relevant and timely. The reported gains on the two public datasets are substantial (e.g., small-lesion mIoU of 77.13 vs. 64.04 for VmUnet on ISIC2018), and the model is comparatively efficient (27.58G FLOPs, 4.64M parameters). The release of code and the use of standard public benchmarks are commendable. However, the formalization of the curriculum learning contribution is incomplete, the loss function is unspecified, and the evaluation lacks statistical validation. These issues currently prevent the paper from being fully reproducible and make the strength of the empirical claims hard to assess.","major_comments":[{"comment":"The curriculum learning objective is not well-posed. Equation (6) is written as an optimization over φ only, with v 'determined by the Difficulty Measurer,' but no initial values or update rule for v are given. In Eq. (7), the first term depends only on l_i and not on v_i, so the only v-dependent term is the quadratic regularizer, which would drive v_i to 0 if v were an optimization variable. The prose says weights are initialized by lesion size and dynamically updated according to loss values, but this is never expressed as a concrete algorithm. Since Tables 3 and 4 attribute measurable gains to '+CL,' this missing specification is load-bearing for the claimed training-strategy contribution. Please provide a rigorous definition of the CL algorithm, including the exact mapping from losses to v, the update schedule, and the role of λ.","section":"Regularized Curriculum Learning Strategy, Eqs. (6)-(7)"},{"comment":"The segmentation loss L(·) used in Eq. (6) and for computing l_i is never identified. The curriculum learning mechanism depends entirely on the loss values, so without the loss definition (e.g., cross-entropy, Dice, or a weighted combination) the reported gains from '+CL' cannot be reproduced. Please state the loss function and any class-weighting or hyperparameters.","section":"Implementation Details"},{"comment":"All quantitative results are single-run point estimates without error bars, confidence intervals, or significance tests. The central claim of improved small-lesion segmentation rests on differences such as 77.13 vs. 64.04 mIoU on ISIC2018, which may be substantial, but the absence of any variance measure makes it impossible to rule out split or initialization effects. Please report mean±std over at least three independent runs, or provide appropriate significance tests, for the main comparisons and ablation studies.","section":"Experimental Results, Tables 1-4"},{"comment":"The tensor notation in the TCMA is underspecified. In particular, Pdi is introduced as a three-dimensional tensor, but Eq. (2) uses tensors P_O, P_I, P_E with channel/category/edge indices whose shapes are not defined, and the relation between the feature pyramid R and these three tensors is unclear. Without explicit tensor dimensions and a precise definition of how the multi-feature tensors are constructed and combined, the spatial-attention component cannot be re-implemented from the paper. Please clarify these definitions.","section":"Proposed Method, Eqs. (1)-(4)"}],"minor_comments":[{"comment":"In Table 1, the SwinUNETR row on ISIC2018 contains a typo, '98..93', for ACC; please correct it to '98.93'.","section":"Table 1"},{"comment":"The related work section contains a missing citation marked as '?' in the list of hybrid models; please insert the intended reference.","section":"Related Work"},{"comment":"References (Wang, Chen, and Zhu 2021a) and (Wang, Chen, and Zhu 2021b) refer to the same survey and should be merged or differentiated.","section":"References"},{"comment":"The category label 'Mam' in Table 1 should be 'Mamba' for consistency.","section":"Table 1"},{"comment":"The complexity analysis states that FLOPs stands for 'Floating Point Operations per Second'; the standard expansion is 'Floating Point Operations' (the rate is FLOPS). Please correct this.","section":"Model Complexity Analysis"},{"comment":"The private Lymph dataset is described only by the number of images and modalities; please add acquisition details (e.g., resolution, patient count, slice thickness) and the exact train/test split, since the largest gains are reported on this dataset.","section":"Datasets"}],"recommendation":"major_revision","confidential_remarks":"The paper would benefit from a stronger statistical evaluation and a fully specified curriculum learning algorithm. If the authors can provide these, the contribution may be acceptable. The private dataset is a concern for reproducibility, but the public results already provide support. The manuscript also has formatting issues suggesting a quick arXiv submission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"At a glance: S3-Mamba is a practical, incremental architecture for small lesion segmentation, and on the two public datasets the small-lesion gains are large and consistent (ISIC small mIoU 77.13 vs VmUnet's 64.04; CVC small 75.40 vs 58.26). Those numbers look real. The paper is worth a serious referee.\n\nWhat's actually new: the EnVSS block (removing the depthwise convolution from VSS, adding SE-style channel attention plus two residual connections) and the TCMA module, which builds a three-way tensor product of input, intermediate, and edge features over multiple scales. That tensor-based attention is a genuinely fresh mechanism, even if the rest is an engineering combination. The ablations show each component contributes, and the complexity analysis is honest.\n\nThe soft spots, in order of weight.\n\nFirst, the curriculum learning section is under-specified to the point of being unreproducible. Eqs. (6)-(7) write min_phi sum v_i l_i + g(v), with v 'determined by the Difficulty Measurer'. But v is not an optimization variable, g(v) contains a rank-based term that does not depend on v, and no update rule for v is given. The prose says the Difficulty Measurer initializes weights by lesion size and updates them based on loss, but there is no equation or pseudocode. Since the ablation attributes measurable gains to +CL, this missing algorithm is load-bearing. The code may resolve it, but the paper as submitted does not.\n\nSecond, the paper presents large-lesion drops without analysis. On ISIC large mIoU is 83.28 vs VmUnet's 84.67; on CVC it is 85.86 vs 87.94. If the goal is small-lesion sensitivity, that trade-off is defensible, but the paper's 'superior' framing is too broad.\n\nThird, no error bars or significance tests. The gains are large enough that they are probably not noise, but for a medical imaging audience, reporting single-run numbers is thin.\n\nFourth, the Lymph dataset is private and supplies the largest reported gains. It is supportive, not verifiable.\n\nCitation pattern is fine; the self-citations are not load-bearing.\n\nRecommendation: send it to peer review. A referee should ask for a precise curriculum learning algorithm (or a downgraded claim), error bars, and a discussion of the large-lesion trade-off. The architecture and the public-dataset results justify the referee time.","headline":"A genuinely useful small-lesion segmentation architecture backed by consistent public-dataset gains, but the curriculum learning formulation is underspecified to the point of being unreproducible from the text.","tokens_in":16782,"tokens_out":3913,"would_cite":true,"duration_ms":34317,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"S3-Mamba reports 77.13% small-lesion mIoU on ISIC2018, beating all compared models.","keywords":["small lesion segmentation","Mamba","state space model","U-Net architecture","tensor-based attention","curriculum learning","multi-scale feature fusion","medical image segmentation"],"falsifier":"Differentiating the stated curriculum objective with respect to the sample weights shows that the squared-weight regularizer forces the minimizer to $v_i=0$ for every $i$, so the training rule that produced the CL ablation columns cannot be the one written in Eqs. (6)-(7); a decisive check is to re-run the ISIC2018 small-lesion split with a fixed lesion-size weighting and no dynamic update, and see whether the 77.13% mIoU survives.","tokens_in":15827,"feed_emoji":"🩺","tokens_out":9116,"duration_ms":69610,"temperature":0.7,"pith_summary":"Small lesions are the earliest visible sign of many diseases, but they occupy only a few percent of an image and are easily washed out by down-sampling in standard segmentation networks. This paper proposes S3-Mamba, a Mamba-based U-shaped model that attacks the problem on three fronts: an Enhanced Visual State Space block that preserves local detail through residual connections and channel reweighting, a tensor-based multi-scale attention that fuses image, prediction, and edge features, and a curriculum-learning training schedule that gradually moves from easy to small, difficult lesions. Across ISIC2018, CVC-ClinicDB, and a private lymph-node MRI dataset, the reported gains are concentrated exactly where the authors intend: small-lesion mIoU on ISIC2018 reaches 77.13%, up from 64.04% for the VM-UNet baseline, and the model reaches 61.19% mIoU on the Lymph dataset. The design principle—preserve fine details rather than model long-range context alone—is what would generalize to other tasks where small structures carry clinical information.","feed_headline":"Mamba redesign hits 77.13% mIoU on small lesions","feed_subtitle":"Residuals, multi-scale tensor attention, and curriculum training lift small-lesion segmentation on three medical datasets.","key_machinery":"The load-bearing machinery is the EnVSSBlock and the TCMA bridge, both organized around keeping multi-scale information alive. EnVSSBlock edits VMamba's VSSBlock by removing the depthwise convolution, adding two residual connections, and inserting a squeeze-and-excitation-style channel reweighting (EnCFBlock); this is what preserves fine detail through the encoder and decoder. TCMA builds three patch scales from embedded input image features, intermediate per-class prediction features, and Sobel edge features, then combines them with a tensor product that sums over patch positions and uses an MLP to produce dynamic weights, finally modulating each decoder feature map. The training strategy adds a regularized curriculum that reweights samples by lesion size and loss, shifting focus from easy to small lesions. Ablations indicate the architecture modules carry most of the improvement, with curriculum learning adding a smaller increment.","core_discovery":"The central claim is that a Mamba-based segmentation network can be made small-lesion-sensitive without sacrificing efficiency by editing the visual state space block itself. Concretely, S3-Mamba removes the depthwise convolution from the VSS block, adds two residual connections, and inserts a squeeze-and-excitation-style EnCFBlock that reweights channels; this is the EnVSSBlock. On top of the U-Net skeleton, TCMA partitions input-image, intermediate-prediction, and Sobel edge features into patches at three scales and computes a tensor-based attention that modulates decoder features, preserving spatial detail at several granularities. A regularized curriculum-learning strategy then reweights training samples by lesion size and loss. The paper reports the top results on the small- and medium-lesion splits of ISIC2018 (77.13% and 81.36% mIoU), strong results on CVC-ClinicDB, and the best numbers on the private Lymph dataset (61.19% mIoU, 75.93% DSC), with ablations attributing the gain to all three components and the full model needing only 27.58G FLOPs and 4.64M parameters.","pith_inferences":["Editorial inference: the stated curriculum objective cannot be minimized as written, so the reader should treat the curriculum's measured contribution as evidence for an unspecified greedy weighting scheme, not for the regularized objective in Eqs. (6)-(7).","Editorial inference: since the architecture components (EnVSSBlock + TCMA) already lift small-lesion mIoU from 64.04% to 74.89% on ISIC2018, most of the reported gain may be attainable without curriculum learning; a direct ablation with EnVSSBlock+TCMA plus plain lesion-size weighting would separate the causes.","Editorial inference: TCMA's use of intermediate prediction features and edge maps ties it to having a reasonably good initial decoder; on extremely low-contrast modalities the predicted-mask signal may be too noisy to provide useful attention, so the method's advantage may shrink when lesions are near-invisible.","Editorial inference: the small/medium/large split is by pixel-percentile within each dataset, so 'small lesion' is relative to dataset distribution, not a clinical size; the model's edge over other methods could change under a clinically defined diameter threshold."],"forward_implications":["On ISIC2018, S3-Mamba reaches 77.13% mIoU for small lesions and 81.36% for medium lesions, beating all nine compared models on the small-lesion split.","On the private Lymph dataset of ultra-small lesions, S3-Mamba reaches 61.19% mIoU and 75.93% DSC, about nine mIoU points above VM-UNet, with visibly fewer false positives in the shown cases.","Ablations on ISIC2018 show small-lesion mIoU climbing from 64.04% at baseline to 70.02% with EnCF+CL, 72.23% with TCMA+CL, 74.89% with TCMA+EnCF, and 77.13% for the full model, so each component contributes.","The full model uses 27.58G FLOPs and 4.64M parameters, making it lighter than VM-UNet (39.77G FLOPs, 6.4M parameters) while scoring higher DSC on Lymph.","The paper attributes the gains to retaining local detail, multi-scale fusion, and size-aware training, not to any single module; the largest architectural jump comes from combining TCMA and EnVSSBlock."],"supporting_citations":[{"why":"It supplies the Visual State Space block and the ImageNet-pretrained VMamba backbone that S3-Mamba's EnVSSBlock modifies and retrains.","marker":"Liu et al. 2024"},{"why":"It supplies the squeeze-and-excitation channel attention that the EnCFBlock inserts into the EnVSSBlock.","marker":"Hu, Shen, and Sun 2018"},{"why":"It is the VM-UNet baseline and main Mamba comparison, and its backbone is the starting point for the ablation study.","marker":"Ruan and Xiang 2024"},{"why":"It provides the U-shaped encoder-decoder skeleton and the U-Net baseline that motivates the down-sampling critique.","marker":"Ronneberger, Fischer, and Brox 2015"},{"why":"It supplies the curriculum-learning framework whose training schedule S3-Mamba regularizes and extends.","marker":"Wang, Chen, and Zhu 2021a"},{"why":"It provides the ISIC2018 dataset and evaluation setup used for the main small, medium, and large lesion comparisons.","marker":"Azad et al. 2019"},{"why":"It provides the CVC-ClinicDB polyp dataset used as the second benchmark.","marker":"Jha et al. 2019"}],"fun_headline_variants":["S3-Mamba: Mamba redesigned for small-lesion sensitivity","Three-pronged Mamba tweak improves small-lesion segmentation","S3-Mamba: 77.13% mIoU on small lesions via triple focus","Channel, spatial, curriculum: S3-Mamba targets tiny lesions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The curriculum-learning contribution depends on an update rule that is not the one written down: minimizing Eqs. (6)-(7) over the sample weights would set every $v_i$ to zero and halt training, so the claimed curriculum gains rely on an unstated different optimization scheme.","fun_headline_variants_meta":{"raw":{"variants":["S3-Mamba: Mamba redesigned for small-lesion sensitivity","Three-pronged Mamba tweak improves small-lesion segmentation","S3-Mamba: 77.13% mIoU on small lesions via triple focus","Channel, spatial, curriculum: S3-Mamba targets tiny lesions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000843,"raw_usage":{"total_tokens":3722,"prompt_tokens":1045,"completion_tokens":2677,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":661,"completion_tokens_details":{"reasoning_tokens":2595}},"tokens_in":661,"tokens_out":2677,"duration_ms":18206,"temperature":1.0,"reasoning_tokens":2595,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T12:08:17.563501+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Differentiating the stated curriculum objective with respect to the sample weights shows that the squared-weight regularizer forces the minimizer to $v_i=0$ for every $i$, so the training rule that produced the CL ablation columns cannot be the one written in Eqs. (6)-(7); a decisive check is to re-run the ISIC2018 small-lesion split with a fixed lesion-size weighting and no dynamic update, and see whether the 77.13% mIoU survives.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It provides the U-shaped encoder-decoder skeleton and the U-Net baseline that motivates the down-sampling critique."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It provides the ISIC2018 dataset and evaluation setup used for the main small, medium, and large lesion comparisons."},{"cited_title":"H.; Riegler, M","cited_arxiv_id":null,"evidence_quote":"It provides the CVC-ClinicDB polyp dataset used as the second benchmark."}],"review_version":1}