{"id":"6724cf85-b20e-4993-b584-61107329921f","arxiv_id":"2505.17905","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"RSS applies GRPO-style reinforcement learning with mIoU rewards to semantic segmentation and claims to beat weakly supervised methods, but the comparison is unfair and the results are unreproducible.","lead":"This paper trains a semantic segmentation network with reinforcement learning rewards instead of pixel-by-pixel labels, using either pixel-level or whole-image feedback. It reports large gains over weakly supervised methods on VOC2012 and Cityscapes, but the image-level reward (mIoU) requires full ground-truth labels, and no code or training details are released.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The image-level reward is mIoU computed from the full ground-truth segmentation map (Sec. 3.2.1), so the claim that RSS 'outperforms existing weakly supervised methods that also rely solely on image-level signals' is invalid and the central 'beyond labels' contribution is unsubstantiated.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: the image-level reward is mIoU, which requires dense pixel ground truth. This is central because the paper's headline contribution is that semantic segmentation can be trained from 'global feedback' or 'image-level signals' and that RSS outperforms weakly supervised methods under that condition. If the reward is mIoU, the training signal is not label-free or weakly supervised; it is a scalar compressed from full pixel labels. The comparison in Sec. 6 to weakly supervised methods is therefore invalid, and the 'first practical application' claim is not demonstrated. Even if the experiments were fully reproducible, this conceptual flaw would remain. Other concerns about missing details or code are secondary; they would not change the verdict. My recommendation is to keep the reader's REJECT verdict unchanged, as the central claim is unsupported by the reported supervision protocol.","tokens_in":11368,"tokens_out":2957,"duration_ms":27509,"concrete_test":"Retrain RSS on VOC2012 replacing the mIoU reward with a reward computed only from image-level class labels (e.g., multi-label classification confidence or CLIP image-text alignment), keeping PSR/PSD and all other hyperparameters identical. If the resulting val mIoU falls below the weakly supervised baselines in Table 5 (e.g., below ExCEL's 78.4), then the reported 82.6/84.7 depends on dense-label-derived rewards and the comparison claim is falsified. Alternatively, an analytical check: verify that the mIoU reward gradient can be computed without accessing per-pixel ground truth; if it cannot, the method is not weakly supervised in the sense claimed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract and Sec. 6 claim that RSS with an image-level reward beats weakly supervised methods that use only image-level signals. However, Sec. 3.2.1 states the image-level reward is 'the mean Intersection over Union (mIoU)'. Computing mIoU requires the dense ground-truth segmentation for every training image. Weakly supervised baselines (CLIMS++, MoRe, ExCEL, etc.) receive only image-level class labels, with no pixel annotations. RSS therefore trains with a scalar derived from pixel-level supervision, while the comparison methods do not. This makes the comparison apples-to-oranges. Moreover, the 'beyond labels' framing collapses: the method does not demonstrate training from genuinely label-free feedback; it simply aggregates dense labels into a scalar reward. The appendix's limitation statement acknowledges mIoU was used instead of real-world feedback, but the headline claims in the abstract and Sec. 1 still assert 'the first practical application' and 'outperforming' weakly supervised methods. Without a reward that truly uses only image-level information, these claims are unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Reward in Semantic Segmentation (RSS), a reinforcement-learning-based framework for training semantic segmentation networks. Two reward granularities are considered: a pixel-level reward that gives each sampled action a per-pixel correctness signal, and an image-level reward that scores an entire sampled action map. To make image-level rewards trainable, the paper introduces progressive scale rewards (PSR) and pair-wise spatial difference (PSD), along with a synchronized advantage normalization (SyncAN) and cold start. Experiments on Pascal Context, VOC2012, and Cityscapes are reported; the authors claim that the image-level variant reaches mIoU within 1.88 points of pixel-level supervised learning on Pascal Context and that it outperforms state-of-the-art weakly supervised segmentation methods that also rely on image-level signals.","tokens_in":11599,"tokens_out":7394,"duration_ms":68984,"significance":"If the central claims held, the paper would offer a meaningful step toward training dense predictors from scalar feedback rather than per-pixel labels, and the two-level reward formulation would be a useful addition to the RL-for-vision toolbox. The empirical results on Pascal Context, in particular the jump from 0.32 to 57.93 mIoU when PSR, PSD, and cold start are combined, suggest that the proposed techniques do address a real optimization difficulty. However, the significance is seriously undercut by two facts: the image-level reward is the mIoU computed from full pixel-level ground truth, so the method is not actually trained from 'beyond labels' feedback; and the comparison with weakly supervised baselines is therefore apples-to-oranges. The paper does not release code or training details, making the claims difficult to verify.","major_comments":[{"comment":"The image-level reward is defined as mIoU ('we utilize the mean Intersection over Union (mIoU) as the image-level reward'). Computing mIoU requires the full ground-truth segmentation map for every training image. The abstract and Section 6 state that RSS 'outperforms existing weakly supervised methods that also rely solely on image-level signals during training'; however, the weakly supervised baselines (CLIMS++, MoRe, ExCEL, DOEI) receive only image-level class labels, with no pixel annotations. RSS therefore trains with a scalar derived from dense pixel-level supervision, while the baselines do not. This is an invalid comparison, and the 'beyond labels' framing collapses. Appendix A.3 acknowledges that mIoU was used instead of real-world feedback, but that does not resolve the issue that mIoU is a deterministic function of the dense label map.","section":"3.2.1 and Section 6"},{"comment":"No training configuration is reported: the optimizer, learning-rate schedule, base learning rate, number of iterations, batch size, input resolution, data splits, number of sampled actions α, the scale list for PSR, and random seeds are all unspecified. Table 3 shows that the last-layer learning-rate factor is critical, but without the base learning rate the result cannot be reproduced. The paper also does not provide code. The empirical results are therefore not independently verifiable, which is a load-bearing problem for a paper whose main contribution is empirical.","section":"4 (Implementation)"},{"comment":"The comparison in Table 5 is not 'under similar conditions': RSS uses ConvNeXt-v2-Nano, a recent strong backbone, while the baselines use ResNet-101, WideResNet-38, or ViT-B. The table does not state which training split was used for RSS or for the baselines, nor does it report the training protocol for RSS on VOC2012. Given that the supervision signal is fundamentally different (dense-label-derived mIoU vs. image-level class labels) and the backbone is different, the claim of 'significantly outperforms' is not a controlled comparison and should be removed or re-framed.","section":"Table 5 and Section 6"},{"comment":"PSD assigns the full reward difference between two sampled action maps as the advantage to pixels where the maps differ, and zero advantage elsewhere. No derivation or bias/variance analysis is provided; the method is presented as a heuristic. Since PSD is one of the two key techniques (with PSR) that make image-level training converge, the paper should at least discuss the conditions under which this advantage estimator is consistent, or explicitly frame it as an empirical trick. The current treatment leaves the reader unable to judge whether the reported convergence is a property of the method or of the particular heuristic.","section":"3.2.3 (Pair-wise spatial difference)"},{"comment":"The claim that RSS is 'the first practical application of reward-based reinforcement learning on pure semantic segmentation' is not supported by the related-work section, which discusses weakly supervised segmentation and RLHF/GRPO but does not survey prior RL-based approaches to segmentation. If such prior work exists, the claim needs qualification; if not, the survey is incomplete. This is a novelty claim that should be substantiated.","section":"1 and 2 (Introduction and Related work)"}],"minor_comments":[{"comment":"The table title contains a typo: 'Leanring rate factor' should be 'Learning rate factor'.","section":"Table 3"},{"comment":"The section title 'rainforment learning' should be 'reinforcement learning'.","section":"2.3"},{"comment":"The text refers to 'Fig. [51]' but should refer to 'Fig. 6'.","section":"7"},{"comment":"The text cites 71.4% mIoU for RSS on Cityscapes without specifying that this is the multi-scale no-cold-start result from Table 6 in the appendix; please clarify the exact configuration.","section":"7"},{"comment":"There are numerous informal and ungrammatical phrases (e.g., 'Let's do the warmup', 'the model nearly does not converge at all'); the paper needs careful editing.","section":"General"}],"recommendation":"reject","confidential_remarks":"The paper's core new idea is an RL-style objective for segmentation with a scalar reward. If the comparison to weakly supervised methods is removed and the mIoU reward is openly acknowledged as a proxy for a future real-world reward, the manuscript could be recast as an empirical study of RL training for dense prediction. As submitted, the abstract and Section 6 make claims that are not supported by the experiments, and the lack of training details prevents verification; I recommend rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear Colleague,\n\nThe punchline: the paper's headline claim is not supported by its own setup. The image-level reward is mIoU computed from the full ground-truth segmentation map (Sec. 3.2.1), so comparing against weakly supervised methods that get only image-level class labels is apples-to-oranges. The abstract's 'beyond labels' framing collapses; the method simply aggregates dense labels into a scalar.\n\nWhat's actually new: applying GRPO-style advantage normalization to semantic segmentation, with two heuristics—progressive scale rewards (PSR) and pairwise spatial difference (PSD)—that address the sparse-reward convergence problem. The ablation table shows mIoU jumping from 0.32 to 57.93 on Pascal Context with image-level reward, which is a real engineering result if it can be reproduced. The PSD idea—using the reward difference between two sampled action maps as advantage for the differing pixels—is simple and possibly useful.\n\nSoft spots, in proportion: the experimental reporting is thin. No training configurations, data splits, or hyperparameters for the main VOC2012 and Cityscapes results; no code release. The comparison to weakly supervised methods is invalid for the reason above. The 'first practical application' claim is unverifiable because no prior RL-for-segmentation work is cited or discussed. The learning-rate factor for the last layer needs to be tuned up to x400, which suggests the method is quite sensitive. The appendix does acknowledge the mIoU reward is a stand-in for real feedback, but that caveat doesn't fix the abstract's claim.\n\nWho it's for: someone working on RL for dense prediction might find the PSD/PSR heuristics worth a look, but the paper needs a major rewrite with honest framing and full details.\n\nRecommendation: I'd desk reject it in its current form—the central comparison is flawed and the missing details make verification impossible. It's a candidate for a resubmission after major revisions.\n\nBest,\n[Your name]","headline":"The paper's central 'beyond labels' claim collapses because the image-level reward is mIoU computed from dense ground truth, yet the PSR/PSD heuristics for training a segmentation network with a scalar reward are a real, if incomplete, contribution.","tokens_in":12106,"tokens_out":2250,"would_cite":false,"duration_ms":19133,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that semantic segmentation can be trained from a single per-image score, reaching within 1.88 mIoU of pixel-supervised training on Pascal Context and beating weakly supervised methods on VOC2012.","keywords":["semantic segmentation","reinforcement learning","image-level reward","weakly supervised learning","advantage normalization","progressive scale rewards","pairwise spatial difference"],"falsifier":"Run RSS image-level on Pascal Context with the mIoU reward replaced by a scalar that is available without ground-truth maps, such as a pre-trained no-reference quality score or a simulator's feedback, keeping PSR, PSD, and cold start fixed; if convergence or the 57.93 mIoU collapses, the paper's central claim that global feedback can replace dense labels is not supported by the current experiments.","tokens_in":11153,"feed_emoji":"🎯","tokens_out":8349,"duration_ms":64024,"temperature":0.7,"pith_summary":"RSS (Reward in Semantic Segmentation) is an attempt to replace pixel-level labels with reward signals in semantic segmentation, at two granularities: per-pixel rewards and a single global image score. At the pixel level, the network samples actions from its predicted class distributions and rewards correct samples with 1 and wrong ones with 0, using synchronized advantage normalization. At the image level, the reward is the mIoU (mean intersection over union) of the whole predicted map; the authors report that naive training fails, while progressive scale rewards (PSR) and pairwise spatial difference (PSD) make it converge to 57.93 mIoU on Pascal Context, within 1.88 of pixel-supervised training, and to 82.6/84.7 mIoU on VOC2012, above recent weak-supervision methods. The contribution is a practical recipe that turns a sparse global score into a dense training signal; if it holds, segmentation (and visual encoders trained through it) could be supervised without dense annotations, using any scalar quality feedback.","feed_headline":"One global score per image trains segmentation to near-label accuracy","feed_subtitle":"Image-level reward reaches 57.93 mIoU on Pascal Context, within 1.88 of pixel-supervised training.","key_machinery":"The central mechanism is the conversion of a global scalar reward into per-pixel advantages via paired stochastic action maps. A sampled action map is a per-pixel class choice drawn from the predicted probability distribution; comparing two such maps, PSD gives the score difference between the two maps as the advantage to pixels that differ between them and zero to pixels that agree, while PSR performs the sampling at several downsampled resolutions so that early training receives rewards from a small action space. SyncAN is a cross-accelerator normalization of those advantages, inspired by synchronized batch normalization, and cold start provides pretrained initialization; together these parts are what make an image-level score usable for convergence.","core_discovery":"On its own terms, the paper claims to be the first practical application of reward-based reinforcement learning to pure semantic segmentation at two granular levels. In the pixel-level setting, categorical sampling of actions per pixel plus GRPO-style advantage normalization (with SyncAN for multi-device training) yields mIoU close to cross-entropy training, 58.92 with cold start versus 59.81 for supervised learning. In the image-level setting, the designed reward is a single scalar per image, computed as mIoU in the experiments; the discovery is that this globally sparse reward becomes trainable when action maps are sampled at multiple downscaled resolutions (PSR) and when the difference in global scores between two sampled maps is assigned as advantage to the pixels where the maps disagree, with zero advantage where they agree (PSD). With these mechanisms, cold-start pretraining, and a large learning-rate multiplier on the last layer, image-level RSS reaches 57.93 mIoU on Pascal Context and, on VOC2012, 82.6/84.7 mIoU, which the paper reads as outperforming weakly supervised methods that use only image-level class labels.","pith_inferences":["Extension not in the paper: the PSD trick of turning paired global scores into dense pixel advantages is a general recipe for sparse-reward dense-prediction tasks, so the same idea could apply to depth estimation, optical flow, or surface-normal prediction when only a global quality score is available.","Editorial: the reported image-level reward is mIoU computed from full ground-truth maps, so the comparison with weakly supervised methods is not strictly apples-to-apples; those baselines receive image-level class labels, while RSS receives a dense-label-derived score. A cleaner test would use a reward that is genuinely obtainable without any segmentation labels.","Testable next step the authors mention but do not run: replace the mIoU reward with a learned no-reference quality model or a simulator's scalar feedback and measure how much of the 57.93 mIoU on Pascal Context survives.","Because the method's convergence depends on paired sampled action maps, it would be interesting to see whether increasing the number of sampled maps per image beyond pairs improves the advantage estimates and reduces the variance of the image-level signal."],"forward_implications":["If the image-level results hold, a semantic segmentation network, and the visual encoder inside it, can be trained from a single score per image instead of pixel-level annotations.","The PSR and PSD pairing is the key to convergence: with the naive image-level pipeline the model stays near 0.32 mIoU, while adding PSR, PSD, and cold start brings it to 57.93 on Pascal Context.","The recipe transfers across architectures: DeepLabv3 with ResNet-101 reaches 44.16 mIoU and Segmenter with EVA02-L reaches 62.37 mIoU using only image-level rewards.","Cold starting from a related dataset has a large effect: on Cityscapes, a Mapillary Vistas cold start raises image-level RSS from about 70.48 to 75.92 mIoU under single-scale evaluation.","Because the reward is a scalar, the paper's stated path is to replace mIoU with real-world feedback or scores produced by a large language model, which would keep the same advantage machinery."],"supporting_citations":[{"why":"Supplies GRPO, the group-relative policy optimization whose advantage normalization the pixel- and image-level reward pipelines are built on.","marker":"[22]"},{"why":"Synchronized batch normalization, the cross-device aggregation idea that SyncAN adapts to advantage normalization.","marker":"[37]"},{"why":"Panoptic feature pyramid networks, the segmentation architecture used (with self-attention as SFPNeXt) for the main ablations.","marker":"[38]"},{"why":"ConvNeXt-v2, the backbone of the SFPNeXt model and of the VOC2012 comparisons.","marker":"[40]"},{"why":"Pascal Context, the dataset where the central mIoU ablations and the near-supervised image-level result are obtained.","marker":"[32]"},{"why":"VOC2012, the benchmark used for the comparison with weakly supervised methods.","marker":"[35]"},{"why":"CLIMS++, a recent image-level weakly supervised method that RSS claims to outperform on VOC2012.","marker":"[44]"},{"why":"DeepLabv3, one of the supervised baselines used to show RSS image-level transfer.","marker":"[41]"},{"why":"Mapillary Vistas, the cold-start dataset that provides most of the Cityscapes gain.","marker":"[53]"}],"fun_headline_variants":["Image-level reward gets within 1.88 mIoU of pixel supervision","Single image-level score trains segmentation near label accuracy","Reward-based segmentation: one global score per image suffices","Image-level reward narrows gap to supervised segmentation","Global reward per image trains segmentation close to labels"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole argument depends on having a score that measures segmentation quality without per-pixel labels, but the experiments use mIoU, which is computed from the full ground-truth map; if no such label-free score is supplied, the claim of training beyond labels collapses.","fun_headline_variants_meta":{"raw":{"variants":["Image-level reward gets within 1.88 mIoU of pixel supervision","Single image-level score trains segmentation near label accuracy","Reward-based segmentation: one global score per image suffices","Image-level reward narrows gap to supervised segmentation","Global reward per image trains segmentation close to labels"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000232,"raw_usage":{"total_tokens":1489,"prompt_tokens":944,"completion_tokens":545,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":560,"completion_tokens_details":{"reasoning_tokens":465}},"tokens_in":560,"tokens_out":545,"duration_ms":6334,"temperature":1.0,"reasoning_tokens":465,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:37:42.575919+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run RSS image-level on Pascal Context with the mIoU reward replaced by a scalar that is available without ground-truth maps, such as a pre-trained no-reference quality score or a simulator's feedback, keeping PSR, PSD, and cold start fixed; if convergence or the 57.93 mIoU collapses, the paper's central claim that global feedback can replace dense labels is not supported by the current experiments.","supporting_citations":[{"cited_title":"Zhang, K","cited_arxiv_id":null,"evidence_quote":"Synchronized batch normalization, the cross-device aggregation idea that SyncAN adapts to advantage normalization."},{"cited_title":"Kirillov, R","cited_arxiv_id":null,"evidence_quote":"Panoptic feature pyramid networks, the segmentation architecture used (with self-attention as SFPNeXt) for the main ablations."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"ConvNeXt-v2, the backbone of the SFPNeXt model and of the VOC2012 comparisons."},{"cited_title":"Mottaghi, X","cited_arxiv_id":null,"evidence_quote":"Pascal Context, the dataset where the central mIoU ablations and the near-supervised image-level result are obtained."},{"cited_title":"Everingham, L","cited_arxiv_id":null,"evidence_quote":"VOC2012, the benchmark used for the comparison with weakly supervised methods."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"CLIMS++, a recent image-level weakly supervised method that RSS claims to outperform on VOC2012."},{"cited_title":"Neuhold, T","cited_arxiv_id":null,"evidence_quote":"Mapillary Vistas, the cold-start dataset that provides most of the Cityscapes gain."}],"review_version":1}