{"id":"24b9598c-fbc0-4c29-ba66-b562a8a4c38f","arxiv_id":"2508.04058","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"TCSAFormer combines token pruning/merging, top-k sparse attention, and a dual-branch convolutional feed-forward network, reporting top Dice/mIoU results on Synapse, ISIC-2018, and CVC-ClinicDB at lower FLOPs.","lead":"TCSAFormer is a medical image segmentation network that reduces computing cost by removing and blending unimportant image pieces before attention, then uses only the most relevant pieces plus a two-branch convolutional block. On three public medical benchmarks it reports the best or near-best accuracy scores while using less computation than many prior transformers.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Ablation's full-model score (79.31) contradicts main result (83.16) on the same Synapse dataset, undermining the reported superiority.","rationale":"The reader's weakest assumption concerned comparability of external baselines in Tables 1 and 2. That is relevant, but the paper's own Tables 3/4 vs Table 1 reveal a more decisive internal contradiction: the full model with all proposed components scores 79.31 in the ablations and 83.16 in the main table on the same dataset. This is not an issue of external reproducibility; it is an inconsistency within the paper's reported experiments. If the ablation protocol differs from the main protocol, then the component contributions (TCP: +0.49 over no compression; DBFFN: +1.60 over MLP-FFN) are measured under conditions that do not match the final model, and their transferability to the 83.16 result is unestablished. If the protocol is the same, then the main result is suspect. Either way, the central claim cannot be evaluated until the discrepancy is resolved. I am not alleging misconduct; this could be an oversight in reporting (e.g., ablations run with fewer epochs or without ImageNet pretraining), but it is a serious omission that must be corrected. The concrete test of reproducing both protocols would settle the issue and is a minimal check before any acceptance.","tokens_in":14324,"tokens_out":6653,"duration_ms":79402,"concrete_test":"Reproduce the full TCSAFormer on Synapse using the exact Section 4.2 protocol, and also using the protocol actually employed for Tables 3/4 (which must be disclosed by the authors). Compare the DSC. If the full model gives ~83 under the main protocol but ~79 under the ablation protocol, then the ablations used undocumented protocol differences and all component conclusions must be re-run under the main protocol. If it gives ~79 under both, Table 1 is misreported. If ~83 under both, Tables 3/4 are misreported or the ablations used a different model variant.","verdict_should_be":"UNVERDICTED","load_bearing_attack":"The ablations in Section 4.4 report the complete TCSAFormer (with both TCP and DBFFN) at DSC 79.31 / HD 23.51 in Table 3 (TCP row) and Table 4 (DBFFN row). Table 1 reports the same architecture achieving DSC 83.16 / HD 17.91 on the same Synapse dataset with the stated protocol in Section 4.2 (224x224, 400 epochs, 0.5CE+0.5Dice, ImageNet init). The gaps of 3.85 DSC and 6.40 HD cannot be explained by random seed variation. This internal inconsistency means either the ablation protocol differs from the main protocol without disclosure (e.g., shorter schedule, different loss, no pretraining), or the main result is not reproducible from the described protocol. Since the ablations are the only component-level evidence and Table 1 is the only SOTA comparison, this discrepancy is load-bearing: it prevents verification of the central 'superior performance' claim and makes the claimed individual contributions of TCP and DBFFN unreliable when transferred to the main configuration.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"TCSAFormer is a U-shaped vision-transformer encoder-decoder for medical image segmentation. It introduces a Compressed Attention (CA) module that prunes and merges tokens, applies a content-aware Top-k sparse attention on the compressed set, and then decompresses to restore spatial positions; it also replaces the standard MLP FFN with a Dual-Branch Feed-Forward Network (DBFFN) using 3x3 and 7x7 depthwise convolutions to capture local and multiscale features. Experiments on Synapse, ISIC-2018, and CVC-ClinicDB report the best or second-best Dice/mIoU among the cited methods, with lower FLOPs claimed in Figure 1. The paper includes ablations comparing token-compression methods and FFN variants.","tokens_in":14599,"tokens_out":6439,"duration_ms":78940,"significance":"If the reported results are reproducible and the comparison is fair, the combination of token pruning, token merging, and per-query Top-k sparse attention would be a useful contribution to efficient medical image segmentation, and the DBFFN design is a sensible modification of the standard FFN. The paper's strengths include experiments on three public benchmarks, component-level ablations against several existing compression and FFN designs, and the explicit aim of reducing FLOPs while retaining accuracy. The authors also follow common practices such as ImageNet initialization and standard loss combinations. However, the current evidence contains a load-bearing internal inconsistency and several experimental-reporting gaps, so the claims cannot be verified as presented.","major_comments":[{"comment":"The main Synapse result (Table 1) reports TCSAFormer at DSC 83.16% and HD 17.91 mm, using the Section 4.2 protocol (224x224, 400 epochs, 0.5 CE + 0.5 DICE, ImageNet init). The full model as reconstructed from the ablations — TCSAFormer with both TCP and DBFFN — is reported as DSC 79.31% and HD 23.51 mm in Table 3 (TCP row) and again in Table 4 (DBFFN row). The gaps of 3.85 DSC and 6.40 HD on the same dataset and the same architecture are too large to attribute to random seed variation. Either the ablation experiments were run under a different protocol that is not disclosed, or one set of numbers is erroneous. Because Table 1 is the central SOTA claim and Tables 3/4 are the only component-level evidence, this inconsistency is load-bearing and must be resolved.","section":"Section 4.3.1 vs Section 4.4, Tables 1, 3, 4"},{"comment":"The training settings in Section 4.2 are given only for TCSAFormer. The paper does not state whether TransUNet, Swin-Unet, HiFormer, MISSFormer, BRAU-Net++, and other baselines were re-trained under the same resolution, image normalization, loss, scheduler, epochs, and data splits, or whether their metrics were copied from the original publications. If the baselines used different protocols, margins such as 83.16 vs 82.85 on Synapse may not reflect a fair comparison. In addition, the claimed lower FLOPs are only visualized in Figure 1 and not tabulated alongside the accuracy numbers with the source/conditions, making the efficiency claim difficult to verify.","section":"Section 4.2, Tables 1 and 2"},{"comment":"All quantitative results are single-run numbers with no standard deviation, no number of seeds, and no statistical significance test. The Synapse margin over MSSAANet (83.16 vs 82.85) is 0.31 DSC, and on several individual organs TCSAFormer is not the best; without variance estimates this margin is not meaningful. The same concern applies to the ablation conclusions: the differences between TCP and PITOME/ToMe (e.g., 79.31 vs 79.22/79.05) are within typical run-to-run variation for medical segmentation, so the claimed superiority of the proposed compression pipeline needs repeated-seed evidence.","section":"Section 4.2, Tables 1 and 2"},{"comment":"The token-merging equations are internally inconsistent. Eq. (7) defines r = n - (rho_m*n) and the text says 'the top r edges with the highest weights are retained,' but the output X_m is in R^{(n-r)xC}. For rho_m=0.3 and n=100, r=70, which exceeds the maximum number of bipartite edges (n/2=50). The notation appears to conflate the number of removed tokens with the number of retained edges. A precise, correctly dimensional definition of rho_m and r is needed to make the method reproducible. Similarly, Eq. (4) defines X_p in R^{HW×C} as a masked version of X, while Eq. (20) uses X_p + Gather(X_d, M); the semantics of Gather and how pruned tokens are restored to their original spatial positions should be stated explicitly.","section":"Section 3.1.1, Eqs. (7)-(11)"}],"minor_comments":[{"comment":"Typo: 'Top-k Sparse Sttention' should be 'Attention'.","section":"Abstract/Introduction"},{"comment":"'The best result is in Blod' should be 'Bold'.","section":"Table 1"},{"comment":"The section headings say 'quantitative comparison' but the figure captions for Figs. 5 and 6 refer to 'qualitative comparison' of segmentation results; consider clarifying 'quantitative evaluation' vs 'visual comparison'.","section":"Section 4.3.2/4.3.3, captions"},{"comment":"Minor typos: 'spacial' should be 'spatial'; 'qualitative comparison' in the CVC-ClinicDB section should be 'quantitative comparison'.","section":"Throughout"},{"comment":"The value of k is defined as lambda*(n-r), which must be an integer; please specify rounding (floor/ceil) and how the Top-k operation handles ties.","section":"Section 3.1.2, Eq. (13)"},{"comment":"The abstract and introduction state that the code is available on GitHub, but no repository URL is provided in the manuscript.","section":"Section 1"}],"recommendation":"major_revision","confidential_remarks":"The internal contradiction between Table 1 and the ablations is the key obstacle. If the authors can show that the ablations were run under a different (disclosed) protocol and reconcile the numbers, the paper may become a solid empirical contribution. If the 83.16% result cannot be reproduced under the stated protocol, the central claim should not be accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The one thing you should know: the paper's own ablation tables contradict its headline result. Full TCSAFormer is reported at 83.16 DSC in the Synapse SOTA table (Table 1), but the same full model in the component ablations scores 79.31 DSC (Tables 3 and 4). That's a 3.85-point gap that random seed variation cannot explain. Either the ablation protocol differs from the main protocol without being disclosed, or the main number isn't reproducible from the described setup. Either way, the central 'superior performance' claim is not supported as written.\n\nWhat's actually new: the exact integration—prune-then-merge token compression, top-k sparse attention, decompression, and a dual-branch convolutional FFN inside a symmetric U-shape—is not present in the cited prior work. The ingredients are all known (DynamicViT, ToMe, KVT, etc.), but this combination is a legitimate engineering contribution. The related work is honest about where the pieces come from, and the FLOPs accounting is useful.\n\nSoft spots, in proportion: the ablation contradiction is load-bearing and not fixable with a footnote. The authors need to run the main model and ablations under one explicit protocol, report standard deviations, and release code (the abstract says GitHub but no link appears). The baseline numbers in Tables 1 and 2 are also presented without saying whether they were re-trained or copied; that's secondary but relevant.\n\nWho this is for: someone designing efficient medical-image transformers might get architecture ideas here, but they should not quote the numbers until the discrepancy is resolved.\n\nRecommendation: this deserves a serious referee, but the referee's first job is to force the authors to reconcile Tables 1 and 3/4. If that can't be done cleanly, the paper should not be published in its current form. I wouldn't desk-reject it outright; the architecture is plausible enough that a careful review could either rescue it or expose it.","headline":"The architecture is a reasonable engineering combination, but the paper's own ablation contradicts its main result: full TCSAFormer scores 79.31 DSC in Tables 3/4 versus 83.16 in Table 1 on the same Synapse benchmark.","tokens_in":15080,"tokens_out":7160,"would_cite":false,"duration_ms":75294,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that combining token pruning and merging before pixel-level top-k sparse attention, plus a dual-branch convolutional feed-forward network, gives medical image segmentation with better accuracy and lower FLOPs than the compa","keywords":["TCSAFormer","Medical image segmentation","Token compression","Sparse attention","Dual-branch feed-forward network","Efficient vision transformer","Top-k sparse attention"],"falsifier":"Retrain all compared baselines under TCSAFormer's exact protocol (same resolution, optimizer, epochs, loss, and data splits) and recompute Dice/HD, mIoU/DSC, and FLOPs; if TCSAFormer no longer achieves the best scores or does not maintain lower FLOPs, the claim of a superior efficiency-accuracy trade-off fails.","tokens_in":14217,"feed_emoji":"🎯","tokens_out":6497,"duration_ms":63863,"temperature":0.7,"pith_summary":"The paper proposes TCSAFormer, a U-shaped vision-transformer network for medical image segmentation that tries to fix two known transformer weaknesses at once: quadratic attention cost and the context-blind MLP feed-forward block. Its answer is to compress tokens before attention — pruning globally irrelevant ones and merging similar ones — then run pixel-level top-k sparse attention on the survivors, and finally decompress back to full resolution; meanwhile it replaces the MLP FFN with a dual-branch depthwise-convolution module that mixes 3x3 and 7x7 contexts. The authors claim this yields the best segmentation scores among the compared transformer methods on Synapse, ISIC-2018, and CVC-ClinicDB with lower FLOPs, i.e., a better accuracy-efficiency trade-off. A reader should care because the method suggests that token compression and sparse attention are complementary rather than competing speed-ups, and that the FFN is a worthwhile place to spend capacity in segmentation transformers.","feed_headline":"83.16% Dice at lower FLOPs: the TCSAFormer recipe","feed_subtitle":"More accurate segmentation with fewer FLOPs than heavy transformer baselines on three medical datasets.","key_machinery":"The Compressed Attention (CA) module: a Token Compression Pipeline (TCP) computes a global semantic token via global average pooling and a linear projection, scores each local token by dot-product, prunes below-threshold tokens, then merges the survivors using a bipartite-graph similarity matching (top-${r}$ edges); a Top-k Sparse Attention (TKSA) then keeps only the ${k}$ most relevant key-value pairs per query on the compressed set; a Token Decompression Pipeline (TDP) unmerges and restores pruned positions via the saved mask and indices, adding them back by a shortcut. The Dual-Branch Feed-Forward Network (DBFFN) uses a $1\\times1$ convolution for channel interaction, two branches of $3\\ti","core_discovery":"The paper's central claim is that medical image segmentation can be made both more accurate and more efficient by (1) compressing tokens before attention — pruning globally irrelevant tokens and merging similar ones — and then (2) applying pixel-level top-k sparse attention to the compressed set, followed by decompression that restores spatial positions. It further claims that replacing the standard MLP feed-forward network with a dual-branch depthwise-convolution network (3x3 and 7x7 kernels) adds the local and multiscale context that segmentation needs. On Synapse, ISIC-2018, and CVC-ClinicDB, TCSAFormer reports the best Dice/mIoU among compared methods (83.16% average Dice on Synapse, 90.","pith_inferences":["Because the pruning and merging ratios are chosen per stage rather than learned, a testable extension is to make them input-dependent, letting the trade-off vary with image clutter.","The same prune-merge-attend pattern could transfer to other dense prediction tasks beyond medical segmentation, such as remote sensing or autonomous driving segmentation, where background tokens dominate.","The Grad-CAM visualizations suggest CA concentrates attention on target regions; if that holds broadly, token compression could double as an attention-regularization mechanism, predicting larger gains on cluttered images with large background areas.","The dual-branch FFN with $3\\times3$ and $7\\times7$ kernels raises the question of whether other kernel-size pairings, or learned kernel sizes, would shift the balance further."],"forward_implications":["Token compression and sparse attention can be combined without sacrificing accuracy, so efficient transformers need not choose between pruning and merging.","Medical image segmentation can reach higher Dice scores — 83.16% on Synapse, 90.23% DSC on ISIC-2018, 94.90% DSC on CVC-ClinicDB — while using fewer FLOPs than the cited transformer baselines.","The dual-branch convolutional FFN is a drop-in replacement for the standard MLP FFN that improves segmentation accuracy while reducing computational cost relative to the baseline.","The per-stage pruning and merging ratios control the accuracy-efficiency trade-off, making the model's cost tunable via those hyperparameters."],"supporting_citations":[{"why":"Supplies the token-pruning strategy that TCP combines with merging.","marker":"[19]"},{"why":"Supplies the token-merging strategy (bipartite matching) that TCP adapts.","marker":"[2]"},{"why":"The strongest compared baseline, using bi-level routing attention, that TCSAFormer claims to outperform.","marker":"[13]"},{"why":"Provides the compared MISSFormer whose ReMix-FFN is the FFN baseline, and whose Synapse data-split protocol is followed.","marker":"[12]"},{"why":"TransUNet baseline and the source of the Synapse train/test split protocol.","marker":"[6]"},{"why":"Swin-Unet baseline and dataset protocol reference for Synapse.","marker":"[5]"},{"why":"Supplies the explicit top-k selection mechanism that TKSA builds on.","marker":"[34]"}],"fun_headline_variants":["Compressed attention: trim tokens, keep segmentation accuracy","TCSAFormer: less compute, higher Dice on medical scans","Token pruning and sparse attention for efficient medical segmentation","Cut FLOPs, lift Dice: token compression in transformers","Efficient medical segmentation via token compression and sparse attention"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The reported superiority over the comparison methods assumes those baselines were evaluated under the same training protocol (resolution, pretraining, loss, schedule, data split) as TCSAFormer; the paper does not describe re-training them for the comparison.","fun_headline_variants_meta":{"raw":{"variants":["Compressed attention: trim tokens, keep segmentation accuracy","TCSAFormer: less compute, higher Dice on medical scans","Token pruning and sparse attention for efficient medical segmentation","Cut FLOPs, lift Dice: token compression in transformers","Efficient medical segmentation via token compression and sparse attention"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000294,"raw_usage":{"total_tokens":1590,"prompt_tokens":829,"completion_tokens":761,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":573,"completion_tokens_details":{"reasoning_tokens":682}},"tokens_in":573,"tokens_out":761,"duration_ms":9441,"temperature":1.0,"reasoning_tokens":682,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T00:54:59.402472+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain all compared baselines under TCSAFormer's exact protocol (same resolution, optimizer, epochs, loss, and data splits) and recompute Dice/HD, mIoU/DSC, and FLOPs; if TCSAFormer no longer achieves the best scores or does not maintain lower FLOPs, the claim of a superior efficiency-accuracy trade-off fails.","supporting_citations":[{"cited_title":"Dynam- icvit:Efficientvisiontransformerswithdynamictokensparsification","cited_arxiv_id":null,"evidence_quote":"Supplies the token-pruning strategy that TCP combines with merging."},{"cited_title":"Token merging: Your vit but faster, in: ICLR","cited_arxiv_id":null,"evidence_quote":"Supplies the token-merging strategy (bipartite matching) that TCP adapts."},{"cited_title":"Missformer: An effective transformer for 2d medical image segmentation","cited_arxiv_id":null,"evidence_quote":"Provides the compared MISSFormer whose ReMix-FFN is the FFN baseline, and whose Synapse data-split protocol is followed."},{"cited_title":"Swin-unet: Unet-like pure transformer for medical image segmentation,in:Europeanconferenceoncomputervision,Springer","cited_arxiv_id":null,"evidence_quote":"Swin-Unet baseline and dataset protocol reference for Synapse."}],"review_version":1}