{"id":"1598507c-ddf0-4d7c-ab60-853099206419","arxiv_id":"2605.28239","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"A reinforced self-evolving framework (L2L) for semi-supervised referring expression segmentation that jointly optimizes the segmentation model and pseudo-labels using multimodal priors and adaptive selection.","lead":"The paper introduces a reinforced self-evolving framework (L2L) for semi-supervised referring expression segmentation that uses multimodal large language model priors and reinforcement learning to refine pseudo-labels from unlabeled image-text pairs. A smart generalist might read it to see how reinforcement learning can help reduce the need for expensive pixel-level annotations in vision-language tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Reinforced pseudo-label selection lacks demonstrated mechanisms to ensure stability in high-dimensional pixel space","rationale":"The reader's weakest_assumption matches the identified load-bearing point exactly. With full text now notionally available, the absence of concrete RL formulation details keeps the claim conditional rather than verified; no other internal inconsistency is visible from the abstract-level description.","tokens_in":1684,"tokens_out":284,"duration_ms":29525,"concrete_test":"Reproduce the L2L training loop on RefCOCO val split; track mean pseudo-label mIoU against held-out ground truth and policy entropy over 10 epochs. If mIoU plateaus or decreases while entropy collapses, or if variance across 3 random seeds exceeds 3 points, the stability assumption fails.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim requires that the exploratory decision process for pseudo-label selection (conditioned on MLLM priors and model predictions) produces stable joint optimization and progressive label improvement. The abstract provides no equations or algorithmic details on state/action spaces, reward function, or policy update rules. In pixel-level RES, any per-pixel or region-level action space is combinatorially large; without explicit regularization against confirmation bias or reward sparsity, the RL loop can amplify early errors rather than enhance reliability. This is the least-secured precondition for the self-evolving loop.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper proposes Learning to Label (L2L), a reinforced self-evolving framework for semi-supervised referring expression segmentation (SS-RES). It uses a multimodal large language model to extract semantic-spatial priors instantiated as initial soft segmentation proposals, which together with textual cues condition a hierarchical segmentation network. Pseudo-label construction is cast as a learnable decision-making process via reinforced pseudo-label selection formulated as an exploratory decision process that adaptively rewards high-utility pixel-level supervision based on multimodal priors and model predictions. This creates a self-evolving loop for joint optimization of the segmentation model and pseudo-labels to improve label reliability under sparse supervision. Experiments on RefCOCO, RefCOCO+, and RefCOCOg report improvements over existing methods.","tokens_in":1783,"tokens_out":572,"duration_ms":29539,"significance":"If the reinforced selection mechanism can be shown to produce stable joint optimization without amplifying early errors, the framework would offer a novel way to address unreliable pseudo-labels in SS-RES by integrating MLLM priors with RL-based selection. This could advance semi-supervised pixel-level language grounding methods. The current presentation, however, provides no equations, state/action definitions, or stability analysis, preventing assessment of whether the central claim holds.","major_comments":[{"comment":"The reinforced pseudo-label selection is described as an 'exploratory decision process' that 'adaptively rewards high-utility pixel-level supervision,' but no equations, state space, action space, reward function, or policy update rules are provided. This formulation is load-bearing for the self-evolving loop claim, yet the combinatorial size of any per-pixel or region action space in RES is unaddressed, leaving open whether stability against confirmation bias or reward sparsity is achieved.","section":"Reinforced pseudo-label selection description"},{"comment":"The abstract and experimental claims state that the loop 'progressively enhancing label reliability' and yields improvements on RefCOCO datasets, but no ablation studies isolate the contribution of the RL selection, no error analysis on pseudo-label quality over iterations, and no verification that gains exceed those from MLLM priors alone. This undermines evaluation of the joint optimization claim.","section":"Experiments and results"}],"minor_comments":[{"comment":"The term 'hierarchical segmentation network' is introduced without architectural details or reference to its base model, making it difficult to reproduce the conditioning mechanism.","section":"Framework overview"},{"comment":"Notation for 'soft segmentation proposals' and 'learnable guidance signals' is introduced without formal definitions or equations, reducing clarity of how MLLM outputs are integrated.","section":"Framework overview"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive comments. We address each major point below and will revise the manuscript accordingly to provide the requested details and experiments.","responses":[{"response":"We agree that the current manuscript lacks sufficient mathematical detail on the reinforced pseudo-label selection. In the revised version we will add a dedicated subsection with explicit definitions of the state space (multimodal prior features concatenated with model predictions), region-level action space (to control combinatorial size), reward function (utility combining prior consistency and prediction confidence), and policy gradient update rules, along with analysis of how the exploratory process addresses confirmation bias and sparsity.","revision_made":"yes","referee_comment":"[Reinforced pseudo-label selection description] The reinforced pseudo-label selection is described as an 'exploratory decision process' that 'adaptively rewards high-utility pixel-level supervision,' but no equations, state space, action space, reward function, or policy update rules are provided. This formulation is load-bearing for the self-evolving loop claim, yet the combinatorial size of any per-pixel or region action space in RES is unaddressed, leaving open whether stability against confirmation bias or reward sparsity is achieved."},{"response":"We agree additional experiments are required. The revision will include ablations that disable the RL selection (replacing it with fixed thresholding), plots of pseudo-label IoU against ground-truth over iterations on a held-out subset, and a direct baseline using only the initial MLLM priors without the self-evolving loop, to isolate the contribution of the reinforced selection to the reported gains.","revision_made":"yes","referee_comment":"[Experiments and results] The abstract and experimental claims state that the loop 'progressively enhancing label reliability' and yields improvements on RefCOCO datasets, but no ablation studies isolate the contribution of the RL selection, no error analysis on pseudo-label quality over iterations, and no verification that gains exceed those from MLLM priors alone. This undermines evaluation of the joint optimization claim."}],"tokens_in":1428,"tokens_out":433,"duration_ms":31749,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The core idea is to treat pseudo-label construction as a learnable exploratory process rather than a static heuristic. It starts with multimodal LLM priors turned into soft proposals, feeds those plus text into a hierarchical segmenter, and then uses reinforcement to pick which pixel-level signals are worth keeping in each round. The self-evolving loop is meant to jointly improve the model and the labels under sparse supervision. That specific combination for referring expression segmentation is the part that looks new.\n\nThe experiments claim gains on RefCOCO, RefCOCO+, and RefCOCOg, which at least shows the authors ran the usual benchmarks and saw movement. For readers who already work on semi-supervised vision-language grounding, this could be a useful data point on whether RL selection helps more than standard pseudo-labeling tricks.\n\nThe weak part is exactly what the stress-test note flags: the abstract gives no state or action definitions, no reward function, and no mention of how they avoid confirmation bias or reward sparsity when the action space is per-pixel. In high-dimensional segmentation, an RL loop can easily lock in early errors instead of correcting them. Without those mechanics shown, the central claim that the reinforced selection produces stable progressive improvement rests on unverified assumptions. The circularity burden looks low because the method brings in external MLLM signals, but that does not fix the missing implementation details.\n\nThis is for people already inside semi-supervised RES or related label-refinement work who need concrete baselines. A reader outside that niche will not get much. It is worth sending to a serious referee because the problem is real and the framing is coherent on its own terms, even though the current write-up leaves the RL component under-specified and the results hard to evaluate without the full experimental section.","headline":"The paper's main move is casting pseudo-label selection as an RL decision process on top of MLLM priors for semi-supervised referring expression segmentation, but the abstract supplies no equations or stability details for that loop.","tokens_in":2283,"tokens_out":440,"would_cite":false,"duration_ms":16620,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Casting pseudo-label selection as reinforcement learning creates a self-evolving loop that jointly improves the model and its supervision signals for referring expression segmentation.","keywords":["semi-supervised segmentation","referring expression segmentation","pseudo-labeling","reinforcement learning","multimodal priors","self-evolving framework","pixel-level language grounding"],"falsifier":"Train the framework for multiple iterations on RefCOCO and measure whether the average IoU of selected pseudo-labels against a held-out ground-truth subset rises steadily or eventually declines.","tokens_in":2576,"feed_emoji":"🔄","tokens_out":619,"duration_ms":25240,"temperature":0.7,"pith_summary":"The paper aims to solve the problem of unreliable pseudo-labels in semi-supervised referring expression segmentation when only limited image-text pairs are annotated. It introduces a framework that extracts semantic-spatial priors from a multimodal large language model to create initial soft proposals and guidance signals for a hierarchical segmentation network. Pseudo-label construction is reframed as an exploratory decision process where reinforcement learning rewards selections that provide high-utility pixel supervision based on both priors and current model predictions. This forms a closed loop in which the segmentation model and the pseudo-labels are optimized together, increasing label reliability over time. Experiments on standard benchmarks show gains over prior semi-supervised methods.","feed_headline":"Reinforced loop refines pseudo labels for referring segmentation","feed_subtitle":"Treating label choice as a learnable decision process improves accuracy when annotations are sparse.","key_machinery":"Reinforced pseudo-label selection as an exploratory decision process that adaptively rewards high-utility pixel-level supervision based on multimodal priors and model predictions.","core_discovery":"The central claim is that reinforced pseudo-label selection, formulated as an exploratory decision process that adaptively rewards high-utility pixel-level supervision based on multimodal priors and model predictions, enables a self-evolving loop for joint optimization of the segmentation model and pseudo-labels, progressively enhancing label reliability under sparse supervision.","pith_inferences":["The same reinforced selection loop could be tested on other pixel-level grounding tasks such as referring video segmentation.","Replacing the MLLM prior extractor with a lighter vision-language model might reduce compute while preserving most gains.","Monitoring the variance of reward signals across iterations could serve as an early indicator of training stability."],"forward_implications":["Joint optimization of model and labels produces progressively more reliable supervision signals.","The framework yields measurable accuracy gains over existing semi-supervised methods on RefCOCO, RefCOCO+, and RefCOCOg.","Multimodal priors from an MLLM can be elevated into learnable guidance that conditions the segmentation network.","The approach maintains generalization when supervision is limited to sparse image-text pairs."],"fun_headline_variants":["Reinforced decisions refine pseudo labels for referring segmentation","Self-evolving loop refines pseudo labels in semi-supervised RES","Reinforced selection refines pseudo labels for expression segmentation","Reinforced self-evolving process refines pseudo labels in RES"],"cache_read_input_tokens":64,"weakest_assumption_plain":"The reinforcement mechanism for choosing pseudo-labels will produce stable improvements rather than unstable or collapsing learning dynamics.","fun_headline_variants_meta":{"raw":{"variants":["Reinforced decisions refine pseudo labels for referring segmentation","Self-evolving loop refines pseudo labels in semi-supervised RES","Reinforced selection refines pseudo labels for expression segmentation","Reinforced self-evolving process refines pseudo labels in RES"]},"model":"grok-4.3","cost_usd":0.009312,"raw_usage":{"total_tokens":4137,"prompt_tokens":609,"num_sources_used":0,"completion_tokens":62,"cost_in_usd_ticks":93124500,"prompt_tokens_details":{"text_tokens":609,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":3466,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":609,"tokens_out":62,"duration_ms":33483,"temperature":1.0,"reasoning_tokens":3466,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-29T12:44:49.780229+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Train the framework for multiple iterations on RefCOCO and measure whether the average IoU of selected pseudo-labels against a held-out ground-truth subset rises steadily or eventually declines.","supporting_citations":[],"review_version":1}