{"id":"be44d9dd-32c3-4a53-a9ce-b97f3a9ec37e","arxiv_id":"2412.11034","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"SAM-IF is an iMTFA-style cosine classifier bolted onto a fine-tuned SAM2 mask generator, and it underperforms iMTFA on COCO 1-shot instance segmentation.","lead":"This paper describes SAM-IF, a method that fine-tunes Meta's SAM2 model to segment specific objects and add new object classes using only a few labeled examples. The authors report results on COCO, but their model scores lower than the existing iMTFA method on every metric.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The few-shot claim is not supported by Table 1: novel AP is 0.5 vs iMTFA's 2.81, and the missing-annotation explanation is unquantified; an oracle re-evaluation is needed.","rationale":"The reader's reject verdict is well supported, and I do not find a reason to overturn it. The paper's own Table 1 is the most direct evidence against the central claim: an incremental few-shot method whose novel-class AP is 0.5 is not 'competitive' unless the evaluation is shown to be misleading. The paper gestures at COCO's missing annotations and fragmented masks, but no experiment isolates those effects. The proposed re-annotation check would settle this. I agree partially with the reader's weakest_assumption: embedding discriminability is a plausible mechanism-level risk, but the decisive issue is evidential — the reported numbers, plus the unquantified post-hoc explanation, leave the central claim unverified. I would keep the verdict at REJECT. I do not weigh the novelty claim heavily; the combination of SAM2 with a cosine-similarity classifier is straightforward, but lack of novelty alone is not the reason for rejection. The missing w_new equation and absence of code are additional reproducibility problems, but the central concern is the metric-evidence gap.","tokens_in":6768,"tokens_out":5554,"duration_ms":52076,"concrete_test":"Re-annotate a random subset of the COCO2014 test images (e.g., 100 images) to add all unlabeled instances of the 20 novel classes, then run the official COCO evaluation script on SAM-IF saved predictions and iMTFA saved predictions on this subset using both original and corrected annotations. If corrected-annotation Novel AP/AP50 for SAM-IF remains near 0.5/1.0, the missing-annotation explanation fails and the central few-shot claim is unsupported; if it rises to iMTFA levels, the reported gap is largely an annotation artifact. Also report the standard deviation across the 10 random 1-shot trials, since that is the stated protocol.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of SAM-IF is that it achieves 'competitive but more reasonable' incremental few-shot instance segmentation. The only quantitative comparison, Table 1, shows SAM-IF below iMTFA on every reported metric, and the gap is largest exactly where the method claims novelty: Novel AP 0.5 vs 2.81 and Novel AP50 1.0 vs 4.72. Section 4.4 attributes this to fragmented SAM masks and missing COCO annotations, but it does not quantify how much of the gap those factors explain. This matters because if the explanation does not hold, the method is simply not performing incremental few-shot learning on novel classes; if it does hold, the headline claim needs to be restated with corrected metrics. Section 5's admission that SAM embeddings are not well suited for classification makes the classifier a plausible bottleneck, but the evidence does not separate classifier error from proposal or annotation error. The missing w_new equation in Section 3.2 further prevents independent verification of the update rule. The burden is on the paper to show the reported low scores are an artifact of evaluation, not a failure of the core mechanism.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces SAM-IF, an incremental few-shot instance segmentation method built on SAM2. It fine-tunes SAM2's image encoder and mask decoder with random point prompts for class-agnostic mask prediction, and attaches a cosine-similarity classifier with a background class for multi-class labeling. For novel classes, class weights are computed from mask embeddings and inserted into the classifier without retraining the decoder. The experiments on COCO2014 in the 1-shot setting compare against iMTFA, reporting lower AP on all metrics; the authors attribute the gap to fragmented masks and missing COCO annotations.","tokens_in":7056,"tokens_out":4208,"duration_ms":38081,"significance":"If substantiated, the approach would be a useful SAM-based alternative for incremental few-shot instance segmentation, with the practical appeal of adding new classes without retraining the segmentation decoder. The paper's strengths include a clear task formulation, the use of random point prompts for class-agnostic mask training, and an ablation showing that training the image encoder helps (Table 2). However, the main claim is not currently supported by Table 1, the central update equation is missing, and no code or variance estimates are provided. The significance therefore remains conditional on a substantial revision.","major_comments":[{"comment":"The stated contribution of 'competitive but more reasonable results' is not supported by the reported numbers. SAM-IF is below iMTFA on every reported metric (Overall AP 17.8 vs 20.13; Base AP 18.1 vs 25.9; Novel AP 0.5 vs 2.81; Novel AP50 1.0 vs 4.72), and the gap is largest on the novel classes that the incremental few-shot mechanism is supposed to enable. Section 4.4 proposes missing COCO annotations and fragmented masks as explanations, but provides no quantification. I request an oracle evaluation (e.g., scoring predicted masks against GT masks, or using GT masks as proposals) and corrected metrics; without it, the paper's central claim is unverified.","section":"Table 1, Section 4.4"},{"comment":"The central update equation for w_new is absent: the text says 'as shown in the following equation' but no equation is printed, so the reader cannot verify how the novel-class weight vector is computed or how the classifier weight matrix W is replaced. This is the core mechanism of the incremental few-shot claim and must be given explicitly.","section":"Section 3.2"},{"comment":"The 10 repetitions of the 1-shot evaluation are reported only as means. Since one shot per class is selected randomly, the result may depend heavily on shot selection; the paper should report mean ± standard deviation or per-run results. No code is released, and with the missing equation in Section 3.2 this makes the experiments very hard to reproduce.","section":"Section 4.1"},{"comment":"The paper acknowledges that SAM embeddings lack structured feature representations needed for classification, especially for fine-grained or novel classes. Given the very low novel-class AP, the paper should include an experiment that separates classifier error from proposal/segmentation error (e.g., feeding GT masks to the classifier) to determine whether the bottleneck is the classifier or the mask proposals. Without this, the explanation in Section 4.4 is not established.","section":"Section 5"}],"minor_comments":[{"comment":"The text says 'The mask encoder produces a low-resolution mask'; this appears to refer to the mask decoder, and the phrase 'cue features' is unclear.","section":"Section 3.1"},{"comment":"The erosion formula M' = M ◦ K = min(sum(M(i,j) over K), 1) is not a standard definition of binary erosion, which is a min over the neighborhood. The formula should be corrected or clarified.","section":"Section 3.3"},{"comment":"Several citations appear inaccurate: [6] is cited as FGN but is a one-shot semantic segmentation paper, and [16] is cited for 'Network-in-Network for better semantic segmentation' but the reference is a general NiN paper. The related-work references need a careful pass.","section":"Section 2.2"},{"comment":"The row 'SAM-IF Base' reports '-' for novel metrics; its purpose and how it differs from the SAM-IF row should be explained in the text.","section":"Table 1"},{"comment":"Figures 1 and 2 are described in the text but are not referenced by number at the points where they are first discussed; please add explicit references.","section":"Figure 1, Figure 2"}],"recommendation":"major_revision","confidential_remarks":"This submission reads like an early technical report rather than a finished journal paper. The missing equation, unsupported 'competitive' claim, and lack of code or variance reporting would need to be addressed before any second round. The topic could fit an incremental-learning venue, but the claims must be corrected and the experiments made reproducible."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is an honest but weak engineering report. The authors take SAM2, bolt on iMTFA's cosine-similarity classifier, and evaluate on COCO. Their own Table 1 shows the method is worse than iMTFA on every metric, with Novel AP of 0.5 vs 2.81. That is not 'competitive but more reasonable'; it is below baseline, and the burden is on the authors to show the gap is an evaluation artifact.\n\nWhat is genuinely new is the specific combination: fine-tuning SAM2 with random point prompts for class-agnostic mask proposals and then using iMTFA's classifier for incremental few-shot class addition. That is a sensible thing to try, and the paper is upfront about borrowing the classifier and update rule from iMTFA. It follows the iMTFA evaluation protocol, repeats the 1-shot selection 10 times, and includes an ablation showing that training the image encoder helps (AP 14.0 to 18.1). That is real work, even if incremental.\n\nThe soft spots are not subtle. The key equation for w_new is missing from Section 3.2; the text says 'as shown in the following equation' and then nothing appears. That alone prevents independent verification. The larger issue is that the headline claim is contradicted by the paper's own numbers. The Section 4.4 explanation (fragmented masks, missing COCO annotations) may be true, but it is not quantified. Without an oracle re-evaluation or at least a per-instance error analysis, the reader cannot tell whether the method learns novel classes at all or just fails at the classifier stage. Section 5's admission that SAM embeddings are not well structured for classification only makes that concern more pointed. No standard deviations, no code release: for a few-shot method, shot variance matters, and ten repetitions without variance is not enough.\n\nThe citation pattern is fine; the authors properly cite iMTFA and SAM2 and don't oversell the novelty in related work. The problem is not dishonesty about provenance; it is a mismatch between the abstract/conclusion and the experimental evidence.\n\nRecommendation: desk reject. There is no load-bearing formal result, no code, and no evidence that the central mechanism works. A serious referee might ask for a corrected evaluation and the missing equation, but that is too much to ask at review time. If the authors redo the evaluation with an oracle or a better classifier, there might be a publishable engineering note here.\n\nReading group: not for me; it is a useful cautionary example but not a paper to learn from.","headline":"The paper is a thin SAM2 plus iMTFA extension whose own results contradict its headline claim; the missing equations and unquantified evaluation gap make it a desk reject.","tokens_in":7543,"tokens_out":4206,"would_cite":false,"duration_ms":34733,"reading_group":"no","serious_thinker":"unclear","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SAM-IF claims that SAM2 can add new object classes from one mask per class by updating only a cosine-similarity classifier's weight matrix.","keywords":["incremental few-shot learning","instance segmentation","Segment Anything Model","SAM2","cosine similarity classifier","class-agnostic segmentation","novel class adaptation","COCO2014"],"falsifier":"Record the mean cosine similarity between class prototypes for held-out novel classes using the trained feature extractor and SAM2 mask embeddings. If the average between-class similarity is close to or larger than the average within-class similarity for any class pair, the weight-insertion rule cannot separate those classes, and the method's novel-class accuracy would collapse. This measurement is a direct check of the paper's core premise.","tokens_in":6582,"feed_emoji":"🎯","tokens_out":6272,"duration_ms":51887,"temperature":0.7,"pith_summary":"SAM-IF claims that a class-agnostic segmenter like SAM2 can be turned into an incremental few-shot instance segmenter by fine-tuning it with random point prompts and attaching a cosine-similarity classifier with a background class. The key move is that new object classes are added by computing a normalized prototype vector from a few mask embeddings and writing it directly into the classifier's weight matrix, leaving the image encoder and mask decoder untouched. If this works, users could teach a segmentation system a new object category from one labeled mask per class, without retraining or storing per-class exemplars. The paper reports 1-shot results on COCO2014 that are competitive with the iMTFA baseline on overall metrics, though with lower absolute numbers, and it identifies the classifier's reliance on SAM embeddings as the main limitation.","feed_headline":"One-shot class additions need only a cosine-weight swap in SAM","feed_subtitle":"Fine-tuned SAM2 plus a cosine-similarity weight update segments novel objects from one example, no decoder retraining.","key_machinery":"The load-bearing object is the cosine-similarity classifier and its weight matrix W, which stores one normalized prototype vector per class. Mask embeddings from SAM2's decoder enter a small feature extractor, are L2-normalized, and are scored against the normalized class vectors in W with a temperature of gamma = 7. For a novel class, the feature extractor converts the few-shot mask embeddings into a vector w_new; this vector is normalized, averaged over the number of shots, and written directly into W. That weight replacement is the entire incremental update, so no gradient step, no decoder retraining, and no storage of the few-shot examples are needed at test time.","core_discovery":"On its own terms, SAM-IF's central discovery is that the incremental few-shot machinery of iMTFA can be transplanted onto SAM2. The paper fine-tunes SAM2's image encoder and mask decoder using randomly sampled foreground points from eroded instance masks, so the model produces class-agnostic instance masks; a separate classifier, trained from scratch, assigns each mask to a base class or to a background class. For novel classes, the method takes the mask embeddings from a few shots, passes them through the feature extractor, normalizes and averages the resulting vectors, and replaces the corresponding rows of the cosine-similarity weight matrix W. This makes category addition a pure weight-update operation. On the 1-shot COCO2014 split, SAM-IF reaches overall AP 17.8 and AP50 27.7; training the image encoder raises base AP from 14.0 to 18.1, and the paper attributes the remaining gap to fragmented masks and COCO's missing instance annotations.","pith_inferences":["The same weight-swap recipe should transfer to any promptable segmenter that emits mask-level embeddings; SAM2 is the test bed, not a requirement of the mechanism.","Because the paper's own future-work section concedes that SAM embeddings lack structured class features, adding a learned transformation between the mask embedding and the cosine head is a natural extension that could lift novel-class accuracy without changing the incremental protocol.","The reported base-versus-novel gap (18.1 AP vs 0.5 AP) suggests the cosine classifier works for familiar classes but nearly fails on unseen ones; a stress test with 2, 5, and 10 shots would show whether averaging more prototypes actually improves separation.","The paper attributes part of the low scores to incomplete COCO annotations and fragmented SAM masks; re-evaluating on a dataset with dense instance annotations could change the numerical comparison with iMTFA."],"forward_implications":["New classes can be added to a deployed segmenter by replacing one weight matrix, so a production system could absorb user-defined categories from a handful of masks without a training run.","Because inference uses uniformly distributed random points rather than user-supplied boxes or points, the system can find target instances automatically once background predictions are discarded.","Few-shot learning collapses to a memory operation: storing one normalized vector per new class extends the classifier, so incremental updates require only the embeddings of the few examples.","The reported 1-shot COCO2014 numbers (overall AP 17.8, AP50 27.7) place SAM-IF near the iMTFA baseline on overall metrics while preserving the ability to add classes incrementally."],"supporting_citations":[{"why":"Supplies the SAM architecture and prompt-based mask generation that SAM-IF adapts.","marker":"[1]"},{"why":"Supplies the SAM2 encoder-decoder that SAM-IF fine-tunes for class-agnostic instance segmentation.","marker":"[2]"},{"why":"Supplies the incremental few-shot instance segmentation framework and cosine-similarity classifier update that SAM-IF borrows.","marker":"[3]"}],"fun_headline_variants":["One-shot class addition: swap cosine weights, skip decoder retraining","Fine-tuned SAM2 gains novel classes via simple weight update","Incremental few-shot segmentation: no decoder retraining needed","SAM-IF: add classes with a cosine-weight swap, not a retrain"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method depends on the assumption that the internal image features SAM2 produces for one object class are similar enough to each other, and different enough from other classes, that comparing them by cosine similarity can separate the classes.","fun_headline_variants_meta":{"raw":{"variants":["One-shot class addition: swap cosine weights, skip decoder retraining","Fine-tuned SAM2 gains novel classes via simple weight update","Incremental few-shot segmentation: no decoder retraining needed","SAM-IF: add classes with a cosine-weight swap, not a retrain"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000392,"raw_usage":{"total_tokens":2010,"prompt_tokens":847,"completion_tokens":1163,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":463,"completion_tokens_details":{"reasoning_tokens":1087}},"tokens_in":463,"tokens_out":1163,"duration_ms":8446,"temperature":1.0,"reasoning_tokens":1087,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T15:21:38.189395+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Record the mean cosine similarity between class prototypes for held-out novel classes using the trained feature extractor and SAM2 mask embeddings. If the average between-class similarity is close to or larger than the average within-class similarity for any class pair, the weight-insertion rule cannot separate those classes, and the method's novel-class accuracy would collapse. This measurement is a direct check of the paper's core premise.","supporting_citations":[{"cited_title":"imtfa: Incremental few-shot instance segmentation","cited_arxiv_id":null,"evidence_quote":"Supplies the incremental few-shot instance segmentation framework and cosine-similarity classifier update that SAM-IF borrows."}],"review_version":1}