{"id":"59decb79-92e6-43dc-98bc-efe2ea1585d7","arxiv_id":"2412.12892","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"SAUGE adapts frozen SAM features through a lightweight side transfer network to produce controllable multi-granularity edge maps, reporting SOTA results on BSDS500 and strong zero-shot transfer.","lead":"The paper presents SAUGE, an edge detector built on a frozen Segment Anything Model (SAM) with a small trainable module that outputs edge maps at multiple levels of detail. It reports state-of-the-art accuracy on BSDS500 and zero-shot generalization to NYUDv2, while letting users choose the desired granularity, which matters for applications needing both coarse boundaries and fine details.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 5's cumulative OR pseudo-labels are not the claimed 'linear blending,' so the multi-granularity outputs may be artifacts of an unprincipled heuristic rather than SAM's inherent granularity.","rationale":"The reader's weakest_assumption already identified the pseudo-label construction in Eq. 5 as load-bearing, and this stress-test concurs while sharpening the issue: Eq. 5 does not actually perform linear blending as the text claims, but rather cumulative ORs. This internal inconsistency makes the concern concrete and testable rather than a general skeptical worry. The proposed check directly isolates whether the label-construction mechanism is responsible for the claimed granularity alignment. The empirical SOTA results could still hold even if this concern is verified, so the appropriate verdict remains CONDITIONAL (the reader's judgment), pending the check. No code is released, which compounds the reproducibility risk, but the paper's methodology is otherwise reasonably standard and the zero-shot NYUDv2 result is reported with a clear protocol. Therefore the verdict should be unchanged: conditional acceptance contingent on the pseudo-label construction being clarified and validated. Agreement with the reader is full because the same underlying assumption is at stake, even though this pass identifies a more specific discrepancy in Eq. 5.","tokens_in":13630,"tokens_out":6757,"duration_ms":69512,"concrete_test":"Re-train SAUGE on BSDS500 using the exact protocol of Table 1, but replace the cumulative-union pseudo-labels of Eq. 5 with a genuine linear blending of the sorted annotations, e.g., Y_alpha = binarize( (1-alpha)*Y^S_1 + alpha*Y^S_N ) for alpha in {0, 0.5, 1}, where alpha=0.5 uses the median annotation instead of the union. Compare the resulting ODS/OIS/AP and the coarse-to-fine ordering of side outputs against the paper's reported values. If the scores remain within roughly 0.01 ODS and the ordering is preserved, the OR vs blend distinction is cosmetic; if stronger deviations appear, then the specific hand-crafted label construction, not an inherent granularity property of SAM features, is what determines the multi-granularity behavior.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that SAUGE 'aligns uncertainty with granularity' by regressing intermediate SAM features to multi-granularity edge maps, supervised by pseudo-labels obtained from 'linear blending' of human annotations. This description does not match Eq. 5. Eq. 5 sorts annotations by edge-pixel count and forms Yc = Y^S_1, Ym = Y^S_1 ∨ Y^S_ceil(N/2), and Yf = Ym ∨ Y^S_N. These are cumulative unions (OR operations), not linear blends: no weights are involved, pixels are added monotonically, and the 'medium' label is simply the union of the two sparsest annotations. Consequently, the pseudo-label granularity is a hand-crafted heuristic, not a normalized interpolation of annotator uncertainty. If the network is trained to fit these cumulative-union targets, the coarse-to-fine ordering and the diversity loss in Eq. 8 are direct consequences of label construction rather than evidence that SAM's intermediate features inherently encode edges at different granularities. This does not automatically invalidate the reported SOTA numbers, but it weakens the conceptual contribution: the claimed 'uncertainty alignment' could reduce to supervised fitting of manually specified labels, and the key insight about SAM features remains unsubstantiated. The discrepancy between the text and Eq. 5, combined with no released code, makes this a concrete correctness risk that should be settled before the scientific claim is accepted.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SAUGE, a lightweight adapter (about 1.5% of SAM parameters) injected into a frozen SAM for multi-granularity edge detection. The Side Transfer Network (STN) progressively fuses SAM's intermediate features to produce coarse, medium, and fine side outputs, which are supervised by pseudo-labels derived from the multiple human annotations. A post-hoc linear combination of the three side outputs yields edge maps at an arbitrary granularity α. The final output is trained with a mask-guided loss using SAM's object masks. Experiments on BSDS500, Multicue, and NYUDv2 report state-of-the-art or competitive results, including a zero-shot evaluation on NYUDv2 using the model trained on BSDS500 and PASCAL VOC.","tokens_in":13961,"tokens_out":4604,"duration_ms":45270,"significance":"If the results hold, the paper makes a useful empirical contribution: it shows that a very small trainable module on frozen SAM can outperform prior dedicated edge detectors while providing granularity control, and its zero-shot generalization to NYUDv2 is notable. The method is clearly specified and the evaluation protocol follows prior work. However, the central conceptual claim—that SAM's intermediate features inherently encode edge granularity and that the pseudo-label construction 'linearly blends' annotations to align uncertainty with granularity—is not supported by the actual equations, and no code is released. The empirical gains are plausible, but the interpretation needs substantial clarification before the scientific claim can be accepted.","major_comments":[{"comment":"The text repeatedly describes the pseudo-label construction as 'linear blending' (Abstract, Introduction, Method), but Eq. (5) defines Y^c = Y^S_1, Y^m = Y^c ∨ Y^S_ceil(N/2), and Y^f = Y^m ∨ Y^S_N, where ∨ is the element-wise OR. This is a cumulative union, not a linear blend: no weights are involved, and the medium and fine labels are obtained by monotonically adding edge pixels from sparser annotations. Consequently, the claimed 'normalization' of granularity and the connection to annotator uncertainty is a hand-crafted heuristic rather than an interpolation of labels. This mismatch is load-bearing because the multi-granularity supervision is central to the method's conceptual contribution. Please either correct the terminology and provide a principled justification for the cumulative-union construction, or compare against a true linear blending of labels (e.g., weighted averages of the binary maps) to show that the proposed construction is not an artifact of the label design.","section":"Method, Eq. (5)"},{"comment":"The paper asserts that 'intermediate SAM features inherently correspond to object edges at various granularities' and that this is the basis for aligning uncertainty with granularity. However, no direct evidence is provided for this correspondence; the STN is trained on the hand-designed pseudo-labels of Eq. (5), so the coarse-to-fine ordering of the outputs may simply reflect the label construction rather than an inherent property of SAM features. To substantiate the key insight, please include a layer-wise analysis (e.g., linear probing of SAM features at different depths against edge maps of different granularity, or an ablation that removes one SAM feature source and shows the granularity ordering degrades). Without such evidence, the claim that SAM provides a strong prior for granularity remains an interpretation rather than a demonstrated mechanism.","section":"Introduction / Method, 'key insight'"},{"comment":"The diversity loss L_diff is designed to make side outputs differ on pixels where the pseudo-labels differ. Because the pseudo-labels are nested (Y^c ⊆ Y^m ⊆ Y^f under Eq. 5), the XOR masks in Eq. (8) simply mark the newly added edge pixels. Thus the 'diversity' is fully predetermined by the cumulative-union construction, and the loss does not independently discover uncertainty structure. This reinforces the concern in the first comment: the diversity of the side outputs is a consequence of the label design. Please clarify what additional role L_diff plays beyond enforcing the already-imposed label hierarchy, and consider an ablation that removes L_diff while keeping the Eq. (5) labels to measure its marginal contribution.","section":"Method, Eq. (8)"}],"minor_comments":[{"comment":"The dataset name 'Muticue' is a typo; it should be 'Multicue'.","section":"Abstract"},{"comment":"'DifussionEdge' is misspelled; the correct name is 'DiffusionEdge'.","section":"Figure 4 and Table 3"},{"comment":"The symbol E_m is used for both the SAM mask embedding (Eq. 1, E^e_m) and the intermediate feature produced by FFB1 (Eq. 2, 'E_m = FFB1(...)'). Please use distinct notations to avoid confusion.","section":"Method, Eq. (2)"},{"comment":"The BSDS500 and NYUDv2 results are reported from a single run without error bars or multiple seeds. Given the small performance gaps over prior methods, please report the variance or at least state that the differences are stable across runs.","section":"Experiments, Implementation Details"},{"comment":"The paper notes the lower AP on Multicue 'may be attributed to overfitting' but provides no supporting evidence. Please either provide a diagnostic (e.g., training/validation AP curves) or temper the claim.","section":"Experiments, Multicue"},{"comment":"The definitions of ψ_j and ω_j are hard to parse: ψ_j uses both Y_mask and Y^mask with similar notation, and the role of the frequency map is unclear. Please clarify the notation and explain the intuition of the guide loss weight in more detail.","section":"Method, Eq. (10)"},{"comment":"No code is released. Given the several non-obvious implementation details (e.g., exact SAM version, prompt grid, label construction), providing code or at least a detailed pseudocode in the supplementary material would substantially improve reproducibility.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper's experimental results are strong and the architectural design is reasonable, but the central conceptual framing is undermined by the mismatch between the text's 'linear blending' and the cumulative-union labels in Eq. (5). This is fixable through clarification and additional analysis, but it is a load-bearing issue for the paper's claimed contribution. I recommend major revision rather than rejection, and I would encourage the editor to ask for the code or a detailed reproducibility appendix, since the small parameter count and zero-shot claim would be much more credible with public code."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Key thing to know: SAUGE is a lightweight SAM adapter that sets new SOTA on BSDS500 and shows strong zero-shot generalization to NYUDv2, with only ~1.5% additional parameters. But the paper's story about 'linear blending' of labels doesn't match the math. Eq. 5 builds coarse/medium/fine pseudo-labels by cumulative OR of sorted annotations, not by any weighted interpolation. That's a real discrepancy and it underpins the 'uncertainty-aligned granularity' claim.\n\nWhat's genuinely good: The Side Transfer Network progressively fuses SAM's intermediate features to produce three multi-granularity side outputs, then linearly combines them for arbitrary granularity. The design is simple and effective. The empirical work is thorough: three datasets, proper evaluation protocol, ablations. The zero-shot NYUDv2 result without any tuning is notable and pushes past prior supervised methods. The qualitative comparisons show real benefit. The paper is well-written and easy to follow.\n\nWhere the soft spots are: The Eq. 5 issue is the biggest. The text repeatedly says 'linear blending,' but the operation is a union. That's not just a wording problem; it affects the interpretation of the diversity loss and the claim that SAM features 'inherently' encode granularity. The coarse-to-fine ordering could be entirely driven by the heuristic pseudo-labels, not by SAM. The authors need to either fix the description, provide evidence that the ordering correlates with annotator agreement, or soften the conceptual claim. Minor issues: no code release, error bars missing on BSDS500/NYUDv2, and Multicue AP is below prior methods (they attribute it to overfitting, but it's still a caveat). The zero-shot claim rests on a single model without hyperparameter tuning; that's fine but limited.\n\nOverall, this is a solid empirical contribution that deserves referee time. I'd recommend conditional acceptance: require the authors to correct the label-construction description, ideally release code, and add an analysis that separates the contribution of the heuristic pseudo-labels from SAM's internal representations. The results are likely real, but the story needs to be honest.\n\nFor your reading group, it's a good case study in how a small mismodeling of labels can be obscured by strong empirical performance.","headline":"Solid empirical adapter on SAM with real SOTA results, but the 'linear blending' label construction is actually cumulative OR — fix the story and the paper holds up.","tokens_in":14465,"tokens_out":2717,"would_cite":true,"duration_ms":25919,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SAUGE claims a frozen SAM can serve as a multi-granularity edge detector, producing edges at any desired granularity with state-of-the-art results on standard benchmarks.","keywords":["edge detection","multi-granularity edge detection","uncertainty modeling","Segment Anything Model","SAM adapter","side transfer network","pseudo-label blending","zero-shot generalization"],"falsifier":"Train the same STN architecture on BSDS500 but shuffle the granularity assignment, e.g., supervise the coarse side output with the finest OR-blended label and the fine side output with the coarsest label; if the model still yields the same side-output ordering and comparable performance, then the multi-granularity behavior comes from the loss structure, not from SAM's features. Alternatively, run a linear probe on each frozen SAM layer against the three pseudo-label levels and check whether coarse-to-fine structure appears in the feature ordering.","tokens_in":13483,"feed_emoji":"🖼️","tokens_out":6370,"duration_ms":57419,"temperature":0.7,"pith_summary":"SAUGE sets out to establish that the longstanding nuisance of inconsistent human edge annotations can be reframed as a granularity axis, and that the intermediate features of the frozen Segment Anything Model (SAM) already contain edge information at multiple granularity levels. The paper builds a lightweight transfer network, about 1.5% of additional parameters, that regresses those frozen features into coarse, medium, and fine edge maps, supervised by pseudo-labels constructed from progressively merged human annotations. This lets the detector output an edge map at any requested granularity, and it reports state-of-the-art scores on standard benchmarks plus zero-shot cross-dataset generalization. If the claim holds, edge detection gains both accuracy and controllable output detail without the cost of training a large task-specific network.","feed_headline":"Frozen SAM plus tiny adapter detects edges at any granularity","feed_subtitle":"A 1.5% parameter adapter on a frozen SAM beats prior edge detectors and dials edge detail from coarse to fine.","key_machinery":"The load-bearing device is the Side Transfer Network (STN), a stack of Feature Fusion Blocks inserted into the frozen SAM. Each Feature Fusion Block applies cross-attention and a gated-convolution feed-forward layer to fuse SAM's shallow features, image embedding, and mask embedding into edge-aware features, which a shared head turns into coarse, medium, and fine side outputs. The arbitrary-granularity output is a linear interpolation of the three side outputs indexed by α, and the supervision includes the OR-blended pseudo-labels from Eq. 5, a pairwise diversity loss among side outputs, and a mask-guided loss on the final fused output.","core_discovery":"The paper's central claim is that SAM's intermediate features—the shallow encoder features, the image embedding, and the mask embedding—carry object-edge information at distinct granularities, and that these can be regressed into three side outputs ordered from coarse to fine. Those three side outputs can then be linearly combined to produce an edge map for any granularity level α in [0,1], where 0 is coarsest and 1 is finest. Supervision comes from sorted human annotations that are progressively OR-combined into coarse, medium, and fine pseudo-labels, together with a diversity loss that forces the side outputs to genuinely differ and a guide loss that uses SAM's own object masks to down-weight confusing pixels. The paper reports that this setup outperforms prior edge detectors on BSDS500, produces competitive results on Multicue, and transfers to NYUDv2 without any fine-tuning.","pith_inferences":["Editorial extension: the method treats granularity as a single scalar axis, which implicitly conflates 'amount of edge detail' with 'degree of annotator disagreement'; it would be worth testing whether α correlates with per-pixel annotator variance.","Editorial extension: because the coarse-to-fine side outputs are supervised by OR-blended labels rather than by any direct measure of uncertainty, the 'uncertainty alignment' could be a descriptive slogan; a clean test is whether the same side-output ordering appears when annotations are shuffled.","Editorial extension: the same STN and pseudo-label recipe could be attached to any frozen vision backbone, and comparing such a variant against SAUGE would isolate how much of the gain comes from SAM's features versus from the multi-granularity supervision itself."],"forward_implications":["A frozen foundation model plus a tiny adapter can replace heavy task-specific training for edge detection.","Users can select edge density at inference time by changing α, with best-matched multi-candidate scores improving as more candidates are sampled.","Because the SAM backbone stays frozen, cross-dataset generalization follows from SAM's priors, and the model transfers to a different domain without fine-tuning.","The pseudo-label construction is independent of the backbone, so the same label-normalization recipe could be reused for other dense prediction tasks with multi-annotator labels."],"supporting_citations":[{"why":"Supplies the frozen SAM backbone whose intermediate features are claimed to encode multi-granularity edge priors.","marker":"Kirillov et al. 2023"},{"why":"Defines the MuGE granularity score and multi-granularity evaluation protocol that SAUGE extends and compares against.","marker":"Zhou et al. 2024"},{"why":"Provides the UAED uncertainty model and the multivariate-Gaussian sampled label construction used for the final output.","marker":"Zhou et al. 2023"},{"why":"Supplies the weighted binary-cross-entropy side-output supervision and the nested-output design that STN's heads follow.","marker":"Xie and Tu 2015"},{"why":"EdgeSAM is the closest SAM-adapter edge detector, serving as the parameter-efficiency and performance baseline that SAUGE must beat.","marker":"Yang et al. 2024"},{"why":"Provides the BSDS500 dataset and its multi-annotator edge labels used for training and evaluation.","marker":"Arbelaez et al. 2010"},{"why":"Provides the NYUDv2 dataset used for the zero-shot cross-dataset evaluation.","marker":"Silberman et al. 2012"},{"why":"Provides the Multicue dataset used for the third benchmark.","marker":"M´ely et al. 2016"}],"fun_headline_variants":["SAM's hidden layers give edge detection adjustable granularity","Frozen SAM plus 1.5% adapter produces edges at any detail level","Uncertainty-aligned edge detection via SAM's intermediate features","Edge detection with tunable granularity using SAM feature regression"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The approach assumes that SAM's internal features genuinely represent edges at different levels of detail and that the progressively merged human labels correspond to those levels; if either is false, the multi-granularity outputs are artifacts of the label construction rather than uncertainty-aligned edge options.","fun_headline_variants_meta":{"raw":{"variants":["SAM's hidden layers give edge detection adjustable granularity","Frozen SAM plus 1.5% adapter produces edges at any detail level","Uncertainty-aligned edge detection via SAM's intermediate features","Edge detection with tunable granularity using SAM feature regression"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000729,"raw_usage":{"total_tokens":3300,"prompt_tokens":1019,"completion_tokens":2281,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":635,"completion_tokens_details":{"reasoning_tokens":2209}},"tokens_in":635,"tokens_out":2281,"duration_ms":13866,"temperature":1.0,"reasoning_tokens":2209,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:36:47.889754+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same STN architecture on BSDS500 but shuffle the granularity assignment, e.g., supervise the coarse side output with the finest OR-blended label and the fine side output with the coarsest label; if the model still yields the same side-output ordering and comparable performance, then the multi-granularity behavior comes from the loss structure, not from SAM's features. Alternatively, run a linear probe on each frozen SAM layer against the three pseudo-label levels and check whether coarse-to-fine structure appears in the feature ordering.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the MuGE granularity score and multi-granularity evaluation protocol that SAUGE extends and compares against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the UAED uncertainty model and the multivariate-Gaussian sampled label construction used for the final output."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the weighted binary-cross-entropy side-output supervision and the nested-output design that STN's heads follow."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"EdgeSAM is the closest SAM-adapter edge detector, serving as the parameter-efficiency and performance baseline that SAUGE must beat."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the NYUDv2 dataset used for the zero-shot cross-dataset evaluation."}],"review_version":1}