{"id":"5614c837-f18d-4fe9-8fe3-e35ba535a7f3","arxiv_id":"2505.14100","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"FSSAM reuses SAM 2's video memory matching for few-shot segmentation by matching query features against pseudo query memories instead of support features, and reports state-of-the-art mIoU on PASCAL-5i and COCO-20i.","lead":"This paper builds a few-shot segmentation system on top of SAM 2, a video foundation model, by first generating rough query masks and then using SAM 2's memory mechanism to refine them. The method reports the best known scores on two standard benchmarks, with a 4.2 point improvement in 1-shot accuracy over the previous best method.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 1 and Tables 3/11 report a 10-point fold-1 mIoU discrepancy (84.9 vs 74.9) for the same final model; the headline 81.0 mean and the 4.2% SOTA margin are unsupported until this is resolved.","rationale":"The reader identified exactly this fold-1 inconsistency (84.9 in Table 1 versus 74.9 in Tables 3 and 11) as one of the two mechanical issues motivating CONDITIONAL. My stress-test confirms it is the most load-bearing concern because the paper's central contribution is a quantitative state-of-the-art claim; if the lower fold-1 value is the true one, the headline mean drops by 2.5 points and the margin over the best baseline drops from 4.2 to about 1.7 points, materially weakening but not necessarily eliminating the SOTA claim. The reader also flagged the selection of IMR iteration count on the test-set curve; I agree that this is a separate, secondary concern, but the fold-1 discrepancy is more decisive because it directly contradicts the reported aggregate score. The recommended verdict stays CONDITIONAL: the paper's design and ablations are still plausible and the issue is fixable by reproducing and correcting one number, but the central claim is not trustworthy in its current form. I am not recommending REJECT because the inconsistency could be a typographical error in one table, and the method's relative ordering over classical and foundation baselines would likely survive even at the corrected mean of 78.5, though with a smaller margin. A verification step is therefore the right path: reproduce fold-1, reconcile the tables, and update the abstract/Section 5.2 claims accordingly.","tokens_in":22169,"tokens_out":2759,"duration_ms":66412,"concrete_test":"Run the released FSSAM code (or request the authors' logs) to reproduce PASCAL-5i fold-1, 1-shot, with the default seed 321, and report the exact fold-1 mIoU. If it is 74.9, Table 1 must be corrected: the mean becomes 78.5 and the margin over FounFSS is about 1.7 points, not 4.2. If it is 84.9, Tables 3 and 11 must be corrected and the ablation/fine-tuning conclusions revisited. Either way, the two reported values cannot both be correct for the same configuration.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative claim is the 1-shot mIoU of 81.0 on PASCAL-5i with fold-1 at 84.9 (Section 5.2, Table 1). However, the identical final configuration appears in Table 3 (the full ablation row, PPG+IMR+SCMA) and Table 11 (FSSAM with fine-tuning) with fold-1 mIoU of 74.9. This is a 10-point internal inconsistency for the same model and the same fold. If 74.9 is correct, the four-fold mean is (81.6+74.9+81.6+76.0)/4 = 78.5, not 81.0, and the claimed 4.2% margin over FounFSS (76.8) shrinks to about 1.7 points. The paper offers no note reconciling Tables 1, 3, and 11. Because the SOTA claim is purely numerical, this unreconciled discrepancy is load-bearing: the headline score and the headline margin each depend on which number is real. A secondary concern is that Table 4 selects the IMR iteration count n=3 as the peak of a test-set curve; n=1 or n=2 would lower the mean to 79.3 or 80.3, further eroding the claimed margin.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FSSAM, a method to adapt SAM 2 for few-shot segmentation. It identifies an incompatibility between SAM 2's same-object video matching and the different-object matching required by FSS, and addresses it with three components: a Pseudo Prompt Generator (PPG) that creates pseudo query masks encoded as query memory, an Iterative Memory Refinement (IMR) that progressively fuses foreground features into the discriminative memory, and a Support-Calibrated Memory Attention (SCMA) that suppresses background features in the memory during cross attention. The method is evaluated on PASCAL-5i, COCO-20i, LVIS-92i, and PASCAL-Part, reporting 1-shot mIoU of 81.0 on PASCAL-5i and 62.3 on COCO-20i, with component ablations, error bars, and qualitative visualizations. The authors also release code and state that the proposed modules introduce no additional learnable parameters.","tokens_in":22469,"tokens_out":4317,"duration_ms":41175,"significance":"If the reported results are correct, the paper makes a useful contribution by showing how a video foundation model (SAM 2) can be adapted to few-shot segmentation through a pseudo-prompt mechanism, rather than treating SAM 2 as a black-box promptable segmenter. The design is transparent, the modules are simple and parameter-free, and the appendices provide error bars, multi-episode testing, model-size studies, and additional datasets, which are all strengths. However, the central quantitative claim is currently undermined by an unreconciled inconsistency in the PASCAL-5i fold-1 score, and by test-set-based selection of the IMR iteration count. These issues must be resolved before the state-of-the-art claim can be trusted.","major_comments":[{"comment":"The same final configuration (PPG+IMR+SCMA, fine-tuned) reports fold-1 mIoU of 84.9 in Table 1, but 74.9 in Table 3 (the full ablation row) and in Table 11 (FSSAM w/ FT). The four-fold mean in Tables 3 and 11 is still reported as 81.0, which is arithmetically impossible with the stated per-fold values (81.6+74.9+81.6+76.0)/4 = 78.5, not 81.0. This inconsistency directly affects the headline claim of a 4.2% margin over FounFSS: if the 74.9 value were correct, the margin would be about 1.7 points. The authors must reconcile these numbers, state which value is correct, and correct the affected means and conclusions. This is a load-bearing issue and cannot be left as an unexplained discrepancy.","section":"Section 5.2, Table 1 vs. Table 3 and Table 11"},{"comment":"The IMR iteration count n is selected as the peak of the test-set performance curve: Table 4 reports n=1,2,3,4 on PASCAL-5i and the paper sets n=3 because it gives the highest mean (81.0). This is test-set model selection: the hyperparameter is chosen using the same benchmark on which the final comparison is made, which can inflate the reported accuracy. The authors should either select n on a validation split, report the results for all values of n in the comparison with baselines, or clearly characterize the sensitivity. The effect is not negligible: n=1 gives 79.3 and n=2 gives 80.3, so the choice of n contributes up to 1.7 points to the headline number. Similarly, the SCMA scaling factor alpha in Eq. (13) is empirically set to 10; the selection procedure for alpha should be described.","section":"Section 5.3, Table 4"},{"comment":"Table 10 is internally inconsistent. For Matcher, the four fold values 37.1, 32.4, 33.7, 38.1 average to 35.3, not the reported 42.9; for FSSAM, the values 34.7, 37.8, 37.2, 41.1 average to 37.7, not the reported 46.4. Moreover, the FSSAM fold values in Table 10 are identical to the first four folds of the LVIS-92i results in Table 9, which suggests that the rows may have been mis-copied or that the mean column corresponds to a different experimental setting. Since the paper uses these results to claim generalizability beyond PASCAL and COCO, the table must be corrected or the experimental protocol clarified.","section":"Appendix C.5, Table 10"}],"minor_comments":[{"comment":"The Pseudo Prompt Generator is described as a contribution, but Equations (2)-(4) follow AENet's FG/BG prototype subtraction almost directly. The authors do cite AENet, but the novelty should be scoped more carefully: the contribution is the use of such priors as pseudo mask prompts for SAM 2, not the prior-generation mechanism itself.","section":"Section 4.1"},{"comment":"The notation AQQ is reused for different quantities: in Eq. (8) it is a similarity map between M_em^FG_Q and a prototype, while in Eq. (12)-(13) it is the query-memory attention map. Please use distinct symbols or clearly distinguish the two roles to improve readability.","section":"Equations (8) and (13)"},{"comment":"The COCO-20i fold-level error bars are fairly large (e.g., fold-0 ranges from 55.8 to 61.8 across seeds), yet the main Table 2 reports only the seed-321 result. Consider reporting the mean over multiple seeds in the main comparison to avoid overstating precision.","section":"Appendix C.1, Table 5"},{"comment":"The ablation table rows labeled only with checkmarks are hard to parse because the column order (PPG, IMR, SCMA) is clear in the header but the rows with two checkmarks do not indicate which one is omitted. Explicitly labeling each row, or adding a note, would improve clarity.","section":"Table 3"},{"comment":"The phrase 'the 1-shot mIoU can be 4.2% better than the best baseline' is stated as a general result, but it refers specifically to PASCAL-5i. Please state the dataset explicitly in the abstract to avoid ambiguity.","section":"Abstract and Section 5.2"}],"recommendation":"major_revision","confidential_remarks":"The fold-1 discrepancy between Table 1 and Tables 3/11 is central and must be resolved. It may well be a typo (the mean 81.0 matches the 84.9 value), but the paper as submitted does not say so, and the reader cannot verify which number is real. The test-set selection of n is also a standard but important methodological flaw that needs a fix, and Table 10's arithmetic errors are serious enough to cast doubt on the appendix's carefulness. The core method and the component ablations are otherwise plausible, and the additional error-bar analysis is commendable. I would be willing to accept after these points are addressed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is the first serious attempt to adapt SAM 2 to few-shot segmentation, and the core idea is genuinely good: SAM 2's memory matching is built for same-object tracking across video frames, while FSS matches different instances of the same class. The authors turn support-query matching into query-query matching by generating pseudo query masks from DINOv2 prototypes, then iteratively refine those memories and suppress background features. The ablations are clean, the modules are parameter-free, and the reported trajectory (69.4 baseline to 81.0 with all components) matches the design story. The appendix is solid: error bars across five seeds, varying episode counts, model-size studies, and an honest limitation section.\n\nThe problem is in the numbers. Table 1 gives the final model's 1-shot fold-1 mIoU as 84.9; Tables 3 and 11 give the identical configuration as 74.9. If 74.9 is right, the mean is 78.5, not 81.0, and the claimed 4.2-point margin over FounFSS shrinks to about 1.7. Table 11 also lists a mean of 81.0 for numbers that average to 78.5, so the arithmetic itself is inconsistent. No note reconciles this anywhere. The SOTA claim is purely numerical, so this has to be resolved before the headline can be believed. The secondary issue is that the IMR iteration count (n=3) is chosen as the peak of a test-set curve in Table 4; n=1 or n=2 would give 79.3 or 80.3, further eroding the claimed margin. That looks like test-set selection, though at least the authors show the full curve.\n\nThese are fixable in revision, and the underlying method is not in doubt: the design story is coherent, the ablations support the individual contributions, and the failure-case discussion in Appendix E is candid. This paper deserves peer review; the referee should ask for corrected tables and an explanation of how n was selected, ideally on a validation split. If the numbers hold, this is a useful baseline and recipe for the FSS community.","headline":"A genuinely good SAM 2 adaptation for few-shot segmentation, but a ten-point fold-1 mIoU discrepancy across tables makes the headline 81.0 and the 4.2% margin unsupported until fixed.","tokens_in":23031,"tokens_out":3009,"would_cite":false,"duration_ms":27752,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":null,"created_at":"2026-08-07T15:40:07.411364+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":null,"supporting_citations":[],"review_version":1}