{"id":"1322a64b-e993-4494-a1bf-e7926df0bb37","arxiv_id":"2501.18753","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"INT improves task-generic promptable segmentation by progressively mining negative candidates, using VLM output differences after masking to select and refine instance-specific prompts.","lead":"This paper introduces INT, a training-free method that improves task-generic promptable segmentation by iteratively selecting instance-specific prompts using VLM score changes before and after masking. It reports gains over prior methods on camouflaged object and medical segmentation benchmarks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central assumption that VLM-output drop magnitude is a monotone correctness signal is untested and can fail through inpainting-mask circularity.","rationale":"The reader's weakest assumption is exactly the one I would stress: the VLM-difference score needs to be a reliable no-GT proxy for prompt correctness. I agree with the conditional verdict. The paper does provide some independent support: a full ablation on CHAMELEON, comparisons against ProMaC, GenSAM, and other baselines, and results across diverse tasks. However, the ablation is on one small dataset, and the central mechanism is not validated against ground-truth prompt correctness. My concrete test would settle whether the mechanism actually identifies correct prompts or whether the segmentation module alone drives the numbers. No change to the reader's CONDITIONAL verdict is needed; the same missing evidence that motivated conditional acceptance is the load-bearing issue. I am not raising a new objection beyond the reader's, and I see no reason to move to a different verdict.","tokens_in":14258,"tokens_out":4466,"duration_ms":47114,"concrete_test":"On COD10K and CVC-ColonDB, where ground-truth labels exist, run INT while logging per image, per iteration: the candidate list from Eq. 2, D(y_k^i) and Dnorm(y_k^i) from Eqs. 3 and 6, the selected A_u^i from Eq. 5, and whether A_u^i matches the ground-truth object name. Repeat with the inpainting mask IM_k^i replaced by the ground-truth mask. If the predicted-mask run selects the true category on fewer than half the images while still matching Table 1, or if the ground-truth-mask run selects true categories substantially more often, then the no-GT selection signal is not reliable and the progressive-mining mechanism is not the source of the reported performance.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that the difference D(y_k^i) in Eq. 3 is large exactly when the candidate category is the true task object, and that the cumulative product in Eq. 7 amplifies stable correct cues over fluctuating wrong ones. This is not established. The masking in Eq. 4 uses the previous iteration's predicted mask IM_k^i; if that mask is wrong, the inpainting removes a region that does not contain the true object. The VLM output can then drop sharply for the wrong category (because a salient distractor was erased) while the true category's score barely moves, because its features were never removed. Eq. 5 picks the largest drop, so the wrong prompt is reinforced, and the normalized weight in Eq. 6 multiplies the wrong class's score in Eq. 7, making the error harder to escape. Figure 1(b) supports the metric only by using ground-truth masks, which are unavailable at test time; no experiment validates monotonicity or the 'stable correct vs. sporadic wrong' assumption on predicted masks. Absent that validation, the reported gains may come from the SAM/CLIP mask generator (Eqs. 8-11) rather than from negative mining.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces INT, a training-free test-time adaptation method for task-generic promptable image segmentation, requiring only a single task-generic prompt such as 'camouflaged animal' or 'polyp'. The method generates candidate instance-specific prompts by processing multi-scale image patches with a VLM, selects the prompt whose category shows the largest VLM output drop after inpainting candidate-indicated regions, and progressively multiplies normalized per-iteration differences to suppress unstable wrong categories. The selected prompt is then used with GroundingDINO, SAM, and CLIP to form a semantic mask, which is fed back as an inpainting guide and image weighting for the next iteration. Experiments on CHAMELEON, CAMO, COD10K, CVC-ColonDB, Kvasir, and ISIC report consistent improvements over ProMaC and other task-generic baselines, with ablations on CHAMELEON attributing gains to each module.","tokens_in":14433,"tokens_out":3964,"duration_ms":37183,"significance":"If the reported gains are reliable, INT provides a practical and conceptually simple extension of promptable segmentation by using VLM response changes as a no-label quality signal for instance-specific prompts. The method is training-free, builds on publicly available components (LLaVA, SAM, GroundingDINO, CLIP, Stable Diffusion), and demonstrates consistent numerical improvements across six datasets and three task types. The paper also makes an explicit falsifiable claim: that progressive multiplication of normalized VLM output differences amplifies stable correct-category cues over sporadic wrong-category cues. That claim is central to the method and is currently supported only by an illustrative figure using ground-truth masks, not by quantitative validation on predicted masks, so the significance of the result depends on additional evidence rather than on the current manuscript alone.","major_comments":[{"comment":"Section 4.1 states that all tasks undergo four adaptation epochs, except polyp segmentation, which uses six. However, the CHAMELEON results in Table 1 (M=0.039, Fβ=0.801, Eφ=0.906, Sα=0.842) are exactly the I=5 row of Table 3(a), not the I=4 row (M=0.045, Fβ=0.792, Eφ=0.903, Sα=0.829). Please clarify which iteration count produced each main result and reconcile the text with the ablation table.","section":"§4.1 vs Table 3(a)"},{"comment":"The text says 'Our results are the average of three trials,' but no standard deviations, confidence intervals, or significance tests are reported in Tables 1–3. Several margins over ProMaC are small (e.g., CAMO M: 0.086 vs 0.090; ISIC Sα: 0.708 vs 0.703), so without variance estimates the reader cannot tell whether the improvements are beyond run-to-run noise. Please provide per-trial results or error bars for at least the main tables.","section":"§4.1"},{"comment":"The central assumption that the magnitude of the VLM output drop after inpainting is a monotone indicator of prompt correctness is not validated on predicted masks. Figure 1(b) is computed with ground-truth masks, which are unavailable at test time, while Eq. (4) uses the predicted mask from the previous iteration. If a predicted mask removes a salient distractor but misses the true object, the distractor category can show a large drop and the correct category a small drop, so Eq. (5) can select the wrong prompt and Eq. (7) can amplify it over iterations. Please provide a direct evaluation of this failure mode, e.g., prompt-selection accuracy against ground-truth categories per iteration, or the fraction of samples where iterative multiplication reinforces an incorrect category.","section":"§3.1, Eqs. (3)–(7)"},{"comment":"The mathematical indexing in the progressive negative mining step is unclear. Equation (6) normalizes D(y_i^k) over the K patches, but the candidate set is described as categories; Eq. (7) then writes D(y_{i+1}^k) = D(y_{i+1}^k) · Dnorm(y_i^k), using the same symbol D for both the raw and updated scores. Please clarify how candidate categories are matched across patches and iterations, and rewrite the update so that the raw difference, the normalized weight, and the cumulative product are distinct variables.","section":"§3.1, Eqs. (6)–(7)"},{"comment":"The final mask is selected as the iteration mask closest to the average of all iteration masks. This rule can discard a genuinely better late-iteration mask that is an outlier relative to the average, and no sensitivity analysis is provided for this choice. Please report how the main results change if the last-iteration mask, or a mean mask, is used instead.","section":"§3.2, Eq. (13)"}],"minor_comments":[{"comment":"There are several typos: 'fliters' should be 'filters' in the Abstract, 'Specfically' should be 'Specifically' in Section 1, 'distintive' should be 'distinctive', and 'propogation' should be 'propagation'.","section":"Abstract and §1"},{"comment":"The label 'Havel' should read 'Halve' to match the text describing horizontal or vertical splitting.","section":"Table 3(b)"},{"comment":"'A V-SAM' should be 'AV-SAM' for the audio-visual SAM reference.","section":"§2"},{"comment":"The venue label 'NeruIPS' in the LLaVA1.5+SAM row should be 'NeurIPS'.","section":"Table 2"},{"comment":"The sentence 'Our study evaluates the ProMaC model’s effectiveness' appears to refer to INT rather than ProMaC; please correct the naming.","section":"§4.1"},{"comment":"The caption 'C k' in Eq. (2) is not defined, and the output structure of the VLM calls (what exactly 'Bk' contains, how many boxes are returned) is left unspecified; please define these quantities.","section":"§3.1, Eq. (2)"},{"comment":"The image weighting hyperparameter w is set to 0.3 without a sensitivity study; even a brief table over a small range of w would help establish robustness.","section":"§3.2, Eq. (12)"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for a computer vision venue and the core idea is interesting, but the acceptance hinges on evidence that the proposed 'output-drop as correctness' assumption works on predicted masks rather than only on ground-truth masks. The internal inconsistency over the number of iterations and the missing error bars are straightforward to fix and should be addressed before the paper is considered further. The comparison group includes weakly supervised methods that use scribble or point labels; that is a strength of the paper, but the authors should make the train/test protocol distinction explicit so readers do not misinterpret the comparison as apples-to-apples."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The genuinely new piece here is the progressive negative-mining rule in Eqs. 3–7: multiplying normalized VLM output differences across iterations to select instance-specific prompts. That specific mechanism is not in GenSAM or ProMaC, and the empirical tables consistently favor INT over ProMaC on all six datasets. The ablations on CHAMELEON also show each component pulling its weight, which is decent evidence that the method is not pure noise. Credit where due: it is a clean, training-free test-time scheme, and the paper reports averages over three trials.\n\nThe soft spots are real but mostly addressable. The biggest one is the central assumption: that a larger VLM output drop when a candidate region is inpainted means the candidate is the true task object. Figure 1(b) motivates this using ground-truth masks, but at test time the inpainting mask comes from the previous iteration's predicted mask. The stress-test concern is legitimate: if that mask is wrong, inpainting can erase a salient distractor, producing a sharp drop for the wrong category, and Eq. 7 then multiplies that wrong signal forward. The paper never validates monotonicity or the 'stable correct vs. sporadic wrong' claim on predicted masks. That is not a refutation—the ablation says removing PNM hurts—but it leaves the mechanism less established than the headline claims.\n\nAlso: the iteration-count protocol is inconsistent. Section 4.1 says four epochs (six for polyps), yet the CHAMELEON result matches the I=5 row of Table 3(a). That smells like tuning on the test set. The Baselines paragraph is copied from the ProMaC paper ('Our study evaluates the ProMaC model's effectiveness...'), which is sloppy and should be rewritten. No code is released, and despite 'average of three trials' there are no error bars or significance tests.\n\nNone of this makes the method wrong. It makes the evidence incomplete. The paper deserves a serious referee, but the referee should push for code, corrected iteration reporting, and at least one experiment that directly tests the negative-mining assumption without ground-truth masks—for example, comparing the VLM-drop signal on predicted masks against random masking, or a per-class failure analysis. If that holds up, this is a useful incremental contribution to promptable segmentation. If not, the reported gains may come more from the SAM/CLIP mask generator than from the negative mining itself.\n\nMy recommendation: accept for peer review, with major revision expected. This is a paper for researchers working on VLM-guided or test-time-adaptive segmentation, and it is worth their time despite the caveats.","headline":"A plausible, incremental extension of the authors' own ProMaC work, with consistent gains on six datasets, but the central VLM-drop assumption is under-validated and a few protocol/presentation issues need fixing before acceptance.","tokens_in":15001,"tokens_out":1879,"would_cite":false,"duration_ms":20791,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A training-free progressive negative mining procedure—erasing candidate objects and multiplying the resulting VLM score drops across iterations—turns one generic prompt into accurate task-specific segmentation without labels, with…","keywords":["task-generic promptable segmentation","instance-specific prompt","negative mining","vision-language models","test-time adaptation","training-free","camouflaged object segmentation","medical image segmentation"],"falsifier":"On a set of images where a large, clearly visible distractor appears next to a small or heavily camouflaged true target, compute $D(y)$ for both candidates under inpainting. If the distractor's score drop exceeds the true target's drop on even a modest fraction of such images, the argmax selection rule (Eq. 5) would pick the wrong prompt; a reader could measure this directly to test whether the assumed monotonicity holds.","tokens_in":54,"feed_emoji":"🔍","tokens_out":6135,"duration_ms":114616,"temperature":0.7,"pith_summary":"Task-generic promptable segmentation asks a model to segment every image in a task—say, 'camouflaged animal'—using only that one text prompt, with no per-image labels. This paper proposes INT, a training-free test-time adaptation method that generates candidate instance-specific prompts from patches of the image, then selects among them by erasing the candidate region and measuring how much the vision-language model's output changes. It claims that multiplying these normalized change scores across iterations, a procedure called progressive negative mining, suppresses unreliable candidates and recovers the correct prompt on difficult images. On six datasets spanning camouflaged objects, polyps, and skin lesions, the paper reports state-of-the-art results against prior promptable segmentation methods.","feed_headline":"Negative mining lets one prompt beat six segmentation benchmarks","feed_subtitle":"Training-free INT erases candidate objects, watches the VLM's response drop, and refines prompts with no labels.","key_machinery":"The load-bearing mechanism is progressive negative mining: for each candidate category, compute $D(y_k^i)$, the maximum drop in the VLM's softmax output when the candidate's predicted region is inpainted; normalize it as $D_{\\mathrm{norm}}(y_k^i)=D(y_k^i)/\\sum_k D(y_k^i)$; then update the next iteration's score by cumulative multiplication, $D(y_k^{i+1})\\leftarrow D(y_k^{i+1})\\cdot D_{\\mathrm{norm}}(y_k^i)$. The candidate with the largest accumulated score becomes the instance-specific prompt. This multiplication is what lets the method suppress once-lucky wrong categories and reinforce the category that consistently loses unique features under masking.","core_discovery":"INT's central claim is that the correct instance-specific prompt is the one whose erasure causes the largest, most stable drop in a VLM's softmax output for that category. The method operationalizes this by hallucinating candidates from overlapping image patches, inpainting each candidate region, and using the normalized difference between pre- and post-inpainting VLM outputs as a selection score; to exploit hard false negative patterns, it multiplies these normalized scores cumulatively over iterations (Eq. 7) so that categories with consistently large differences dominate while sporadic errors decay. The selected prompt drives a semantic mask generator that combines GroundingDINO detections, SAM masks, and CLIP semantic scores, and the resulting soft mask is fed back to focus the next prompt iteration. The paper claims this makes a single task-generic prompt sufficient for accurate segmentation on camouflaged-object and medical datasets, reporting improvements over the prior state of the art on all six evaluated benchmarks.","pith_inferences":["A natural extension would use the accumulated negative-mining score as a per-image confidence signal, flagging low-confidence cases for human review; the paper does not report such a use.","The method's behavior likely depends on the inpainting model's quality: if inpainting fails to plausibly fill a region, the measured output drop may reflect artifacts rather than the absence of the object, and testing with weaker inpainting would clarify how much of the gain comes from the mining rule versus the erasure tool.","Whether progressive multiplication helps or hurts probably hinges on the first iteration: a wrong candidate that scores large at $i=1$ gets amplified, so perturbing or randomizing the initial candidate ranking could reveal how robust the correction is.","If the assumption generalizes beyond natural and medical images, the same machinery could be applied to other VLM-driven dense prediction tasks such as referring expression segmentation, though the paper does not test this."],"forward_implications":["A single generic prompt (\"camouflaged animal\", \"polyp\", \"skin lesion\") yields per-image, instance-specific segmentation without any manual annotation on six datasets.","Erroneous instance-specific prompts produced early in adaptation can be corrected in later iterations, reducing error propagation that earlier VLM-based promptable segmenters suffer from.","Because the pipeline is training-free, it can be applied to a new task or dataset at test time with no per-dataset optimization.","The method can locate and segment multiple task-related objects within one image, something the paper notes earlier methods could not reliably do.","On camouflaged-object benchmarks, INT reports results above both point- and scribble-supervised weakly supervised methods, despite using no training labels at all."],"supporting_citations":[{"why":"Defines the task-generic promptable segmentation setting and the Spatial CLIP module that INT uses to map prompts to image regions.","marker":"[Hu et al., 2024a]"},{"why":"Introduces hallucination-driven candidate generation by processing image patches; INT builds its candidate lists on this approach.","marker":"[Hu et al., 2024b]"},{"why":"Provides the SAM mask generator that turns prompts and boxes into segmentation masks.","marker":"[Kirillov et al., 2023]"},{"why":"GroundingDINO detects candidate bounding boxes for the instance-specific prompt and also serves as a baseline.","marker":"[Liu et al., 2023c]"},{"why":"CLIP supplies the semantic similarity scores used to weight and refine masks toward the instance-specific prompt.","marker":"[Radford et al., 2021]"},{"why":"LLaVA-1.5 is the vision-language model whose output differences drive negative mining.","marker":"[Liu et al., 2023b]"}],"fun_headline_variants":["One prompt, six benchmarks: INT's negative mining wins","Erase to select: INT mines negatives for sharper segmentation","INT finds the right prompt by removing the wrong ones","Negative mining sharpens prompts for universal segmentation","Promptless? No: INT uses negative mining to nail any segmentation"],"cache_read_input_tokens":17152,"weakest_assumption_plain":"The assumption that carries the method is that erasing the true target always makes the vision-language model's score for that target drop more than erasing any distractor, and that this ordering remains consistent across iterations.","fun_headline_variants_meta":{"raw":{"variants":["One prompt, six benchmarks: INT's negative mining wins","Erase to select: INT mines negatives for sharper segmentation","INT finds the right prompt by removing the wrong ones","Negative mining sharpens prompts for universal segmentation","Promptless? No: INT uses negative mining to nail any segmentation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000629,"raw_usage":{"total_tokens":2909,"prompt_tokens":949,"completion_tokens":1960,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":565,"completion_tokens_details":{"reasoning_tokens":1881}},"tokens_in":565,"tokens_out":1960,"duration_ms":11829,"temperature":1.0,"reasoning_tokens":1881,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T22:37:03.103807+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a set of images where a large, clearly visible distractor appears next to a small or heavily camouflaged true target, compute $D(y)$ for both candidates under inpainting. If the distractor's score drop exceeds the true target's drop on even a modest fraction of such images, the argmax selection rule (Eq. 5) would pick the wrong prompt; a reader could measure this directly to test whether the assumed monotonicity holds.","supporting_citations":[],"review_version":1}