{"id":"f21df1e1-0479-4e2e-ac3a-057f2cfbf9e8","arxiv_id":"2506.06748","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"On the EPIC-KITCHENS VISOR test set, the proposed Cutie-based egocentric video object segmentation method with SAM2-pretrained Hiera-L and Depth Anything V2 fusion reports a J&F score of 90.1%.","lead":"A team reports an egocentric video object segmentation system that combines SAM2's pretrained visual features with depth cues from Depth Anything V2 inside a Cutie memory-based framework, reaching 90.1 J&F on the EPIC-KITCHENS VISOR test set. The report is a competition entry with ablations on the VISOR val set showing each component adds a small gain.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The depth contribution to the 90.1% J&F is supported only by a single +0.4 val-set ablation with no error bars and no test-set depth-off comparison; the claimed causal role of depth is not established.","rationale":"The strongest claim is that the system reaches 90.1% J&F and that depth is an important part of the recipe. The paper's ablations isolate two pretraining effects (MAE→SAM2: +1.4 val) and depth (+0.4 val) plus post-processing (+0.2 val). The SAM2 effect is large and plausible; the depth effect is the fragile link: it is one run, no confidence interval, no test-set replication, and only slightly larger than typical seed-level variation in VOS fine-tuning. It is not internally contradictory, and the positive val direction is real evidence in the paper's favor, so this is not a rejection. The condition to accept is to demonstrate that the depth ablation is reproducible and, if possible, to report a no-depth test variant. That is exactly what the reader's CONDITIONAL verdict already asks for.","tokens_in":4889,"tokens_out":6446,"duration_ms":70181,"concrete_test":"Re-run the Table 2 depth ablation (Hiera-L/SAM2 with versus without DA V2 depth, same hyperparameters) for three seeds and compute a paired, per-video bootstrap 95% CI on the J&F difference. If the CI includes 0 or the effect is not consistently above 0.3 J&F, the +0.4 depth gain is within training noise and the central claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that fusing Depth Anything V2 geometric features with SAM2/Hiera in Cutie reaches 90.1% J&F and that depth is a substantive ingredient—rests on Section 4.2, Table 2. The only depth-vs-no-depth comparison is a single +0.4 J&F gain on VISOR val (88.3% without depth versus 88.7% with depth; 88.9% after post-processing), from one training run with no error bars, per-video intervals, or seeds. Table 1, the test-set comparison, varies only Flip and MS; it never reports the 90.1% result without the depth stream. The +0.4 gain is smaller than the +0.4 test gain attributed to multi-scale inference alone, so it is not obvious that the depth stream survives the val-to-test domain shift. The paper is internally consistent, but the abstract's causal statement 'depth-based geometric cues' cannot be checked from the reported numbers because the depth contribution is unreplicated and unquantified on the test set.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents the THU-Warwick submission to the EPIC-KITCHEN VISOR 2025 semi-supervised video object segmentation challenge. The method builds on the Cutie baseline, replaces the visual backbone with a Hiera-Large encoder initialized from SAM2, and adds a geometric encoder from Depth Anything V2 (DINOv2-Large with a DPT decoder) whose multi-scale features are concatenated with visual features and fused by a learnable MLP. Training is two-stage: initialization on the Mega VOS collection with frozen encoders, then fine-tuning on the VISOR train and val sets. The paper reports a J&F of 90.1% on the VISOR test set with multi-scale and flip inference, and ablations on the val set showing gains from SAM2 initialization, depth features, and post-processing.","tokens_in":5067,"tokens_out":4189,"duration_ms":45218,"significance":"If the reported results hold, the paper demonstrates that combining large-scale visual pretraining (SAM2) with geometric depth cues (Depth Anything V2) is beneficial for egocentric video object segmentation, a domain characterized by clutter, occlusion, and rapid viewpoint change. The method is built from publicly available components and the ablation structure is sensible, which makes the findings potentially reproducible and useful for the challenge community. However, the significance is tempered by the absence of error bars or repeated runs, the lack of a test-set ablation that removes the depth stream, and the sparing description of the fusion architecture.","major_comments":[{"comment":"The only evidence for the depth contribution is a single +0.4 J&F gain on the val set (88.3% without depth versus 88.7% with depth). No error bars, multiple seeds, or per-video breakdowns are reported, and Table 1 never evaluates the test-set configuration without depth. Because a +0.4 gain is exactly the magnitude of the MS+Flip gain on the test set (90.1% versus 89.7%), it is plausible that the depth gain is within run-to-run variance. This is load-bearing because the abstract and introduction attribute the method's success to \"depth-based geometric cues.\" Please provide a test-set ablation without the depth stream, or repeated training runs with variance estimates, to establish that the depth contribution is real and not an artifact of a single run.","section":"4.2 (Table 2)"},{"comment":"The fusion module is underspecified. The text says visual and geometric features are \"concatenated at matching scales and subsequently fused via a learnable MLP layer,\" but it does not state the MLP architecture (depth, hidden width, normalization, activation), whether the three scales are fused independently or jointly, or how the resolution mismatch from different patch sizes is exactly resolved (\"we resize the input frames fed into the geometric encoder\" is ambiguous regarding the resizing rule). Additionally, the training schedule for the fusion module—whether it is trained in stage 1, stage 2, or both, and whether any encoder weights are unfrozen during VISOR fine-tuning—is not specified. Since the depth-fusion design is the paper's primary novelty, these details are necessary for reproducibility and for assessing whether the reported gain comes from the fusion mechanism or merely from added parameters.","section":"3"},{"comment":"The test-set ablation is incomplete. Both rows in Table 1 include Flip, so there is no condition with neither Flip nor MS, meaning the individual contribution of Flip and MS on the test set cannot be separated. Moreover, the caption of Table 2 introduces \"Post\" without defining it, and the sentence \"The highlighted scores and the bottom results in Table 1 come from the same setting\" is confusing: Table 1 reports test-set scores (90.1%) while Table 2 reports val-set scores (88.9%), and the training iterations differ (100k versus 50k). Please define \"Post\" explicitly and clarify what setting is shared between the two tables.","section":"4.1 (Table 1)"},{"comment":"The comparison with \"fine-tune the original SAM2 model directly on VISOR\" (J&F 87.8%) is used to argue that \"effective memory design for complex backgrounds and long-term dynamics is still crucial.\" However, this comparison conflates several factors: the SAM2 memory mechanism differs from Cutie, the SAM2 baseline does not include the depth stream or the two-stage training, and the number of training iterations may differ. The proper control for isolating SAM2 initialization is the Hiera-L (MAE) row in Table 2, which the paper does include. Please restrict the claim to what the comparison actually supports, or add an ablation that varies only the initialization within the Cutie framework.","section":"4.2"}],"minor_comments":[{"comment":"The phrase \"aJ&Fscore\" appears twice with a missing space; the correct form is \"a J&F score.\"","section":"Abstract and Introduction"},{"comment":"The notation for feature maps, \"{f si v}3 i=1\" and \"{f si g}3 i=1\", is not typeset clearly in the provided text; please use consistent subscripts and superscripts (e.g., f_v^{s_i}) to avoid ambiguity.","section":"3"},{"comment":"The term \"max skip as 1 when sampling frames as pseudo-training videos\" is not defined in this paper; please provide a brief explanation or a reference to the Cutie paper where this hyperparameter is described.","section":"4.1"},{"comment":"Several references have incomplete or inconsistent bibliographic information, such as [3] and [11] lacking a venue or conference name, and [13] not specifying the venue. Please standardize the reference list.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"This is a concise challenge report rather than a full research paper, and the authors should be evaluated in that context. The main technical novelty—depth fusion—is weakly supported by a single val-set ablation without error bars or a test-set depth-off comparison. If the authors can provide a test-set ablation or repeated runs with variance estimates, and flesh out the fusion architecture details, the paper would be acceptable for a proceedings or workshop venue. There is no indication of circularity or internal inconsistency; the concern is purely about the strength of evidence for the depth contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a clean, modest challenge report. The new thing is combining Cutie's memory framework with a SAM2-pretrained Hiera-Large encoder and Depth Anything V2 features, and showing 90.1 J&F on the VISOR test set. The SAM2 pretraining clearly helps: it lifts val J&F from 86.9 to 88.3. That is a real, internally consistent result, and the paper is honest that the gain comes from known components rather than a new mechanism.\n\nThe soft spot is the depth claim. The only depth-vs-no-depth comparison is a single run on val, +0.4 J&F (88.3 to 88.7, 88.9 with post-processing). There are no error bars or multiple seeds, and the test-set ablation only varies flip and multi-scale inference—depth is never removed on test. The +0.4 val gain from depth is the same size as the +0.4 test gain from multi-scale alone, so the causal statement in the abstract—that depth-based geometric cues drive the result—is not actually checkable from the reported numbers. It may well be true, but this report does not establish it.\n\nThe other gaps are typical for this format: no code, sparse details on the fusion MLP, and no external comparisons. For a workshop/challenge report that's tolerable, but it limits what a referee can verify.\n\nWho's this for? Anyone working on egocentric VOS, especially challenge participants who want a solid recipe. It is not a methodological breakthrough and won't change how people think about the problem. But it is a competent engineering result with an honest ablation table.\n\nMy recommendation: don't desk-reject it. Send it to review with a request for error bars (or at least multiple seeds) and a test-set depth-off ablation. If the authors can show depth survives on test, the paper is a perfectly good short contribution. If not, they should soften the abstract's causal language. Either way it's a legitimate, if minor, empirical data point.","headline":"A competent but incremental VISOR challenge report; the SAM2-pretraining gain is solid, while the depth contribution is only supported by a single val ablation and should be treated as unverified.","tokens_in":5659,"tokens_out":2274,"would_cite":false,"duration_ms":23207,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A SAM2-initialized visual backbone fused with depth features reaches 90.1% J&F on the EPIC-KITCHENS VISOR test set.","keywords":["Semi-supervised video object segmentation","Egocentric vision","EPIC-KITCHENS VISOR","Depth Anything V2","SAM2","Cutie","Multi-scale feature fusion","Long-term memory"],"falsifier":"Re-run the validation ablation with the Depth Anything V2 branch fed scrambled or heavily blurred depth maps while keeping all other settings identical; if J&F stays at 88.7% or higher, the reported depth contribution comes from the fusion module's parameters rather than from geometric information.","tokens_in":4632,"feed_emoji":"🎯","tokens_out":4688,"duration_ms":44838,"temperature":0.7,"pith_summary":"This paper reports a semi-supervised video object segmentation system for egocentric video, aimed at the EPIC-KITCHENS VISOR benchmark, where the targets are hands and active objects. The authors argue that visual pretraining from SAM2 and geometric cues from a depth model are complementary: SAM2 supplies strong object perception and frame-to-frame correspondence, while depth helps separate targets from cluttered backgrounds and track through occlusions. They integrate both in the Cutie long-term memory framework, fusing multi-scale Hiera-Large visual features with Depth Anything V2 features through a learnable MLP. The method reaches a J&F of 90.1% on the VISOR test set, with ablations attributing roughly 0.4 J&F on validation to the depth branch and a larger gain to SAM2 initialization.","feed_headline":"Depth cues lift SAM2 video segmentation to 90.1 percent","feed_subtitle":"Adding Depth Anything V2 features to a SAM2-initialized Cutie baseline beats fine-tuning SAM2 on egocentric VISOR.","key_machinery":"The central object is a dual-encoder fusion inside the Cutie video object segmentation framework. The visual encoder is Hiera-Large, a hierarchical vision transformer whose parameters are initialized from SAM2, and the geometric encoder is the Depth Anything V2 network, whose DINOv2-Large backbone and DPT decoder produce multi-scale depth-derived embeddings. Features from the two encoders are extracted at 1/4, 1/8, and 1/16 resolution, resized to handle patch-size mismatches, concatenated scale-wise, and fused by a learnable MLP before entering Cutie's memory readout and mask decoder. This lets the model add geometric context as a trainable side channel while keeping Cutie's object-aware long-term memory.","core_discovery":"On the VISOR test set, the segmentation pipeline reaches a J&F score of 90.1% (J 88.1%, F 92.0%) when the Hiera-Large visual encoder is initialized with SAM2 weights, depth features from Depth Anything V2 are fused into the Cutie memory framework, and multi-scale and flip inference are used. On the VISOR validation set, switching the visual backbone from MAE-initialized Hiera-Large to SAM2-initialized Hiera-Large raises J&F from 86.9% to 88.3%, adding the geometric encoder raises it further to 88.7%, and multi-scale-plus-flip post-processing raises it to 88.9%. The paper interprets these results as evidence that large-scale visual pretraining and geometric depth cues jointly improve egocentric video object segmentation, and that long-term memory design still matters because directly fine-tuning SAM2 reaches only 87.8% J&F.","pith_inferences":["Depth cues may help most precisely in frames with hand-object occlusions and near-field objects; a per-clip breakdown of the depth ablation on VISOR's interaction-heavy segments would test this.","The same dual-encoder fusion could generalize to other egocentric tasks such as active-object detection or affordance segmentation, where depth disambiguates physical contact.","Because the fusion module is a small trainable MLP, the architecture leaves room for swapping in other geometric estimators or adding temporal depth consistency as a training signal.","If depth features mainly serve as a shortcut for static-scene structure, the depth gain should be larger on background-cluttered clips and near zero on clips where the camera stays still."],"forward_implications":["If the 90.1% test score holds, combining a SAM2-pretrained visual backbone with depth cues is a strong recipe for egocentric semi-supervised video object segmentation on VISOR.","The 1.4-point validation gain from SAM2 initialization over MAE initialization implies that large-scale visual pretraining transfers well to hand-and-object tracking.","The roughly 0.4-point depth gain supports adding geometric encoders to memory-based video object segmentation frameworks when scenes are cluttered and occlusions are frequent.","Multi-scale and flip fusion add further J&F, so test-time aggregation remains useful even with foundation-model features.","Directly fine-tuning SAM2 underperforms the Cutie-based design, indicating that memory architecture still contributes beyond backbone strength."],"supporting_citations":[{"why":"Cutie is the baseline framework that provides the long-term object-aware memory mechanism being extended.","marker":"[4]"},{"why":"SAM2 supplies the pretrained visual backbone weights and also serves as the directly fine-tuned comparison baseline.","marker":"[18]"},{"why":"Depth Anything V2 provides the geometric encoder and multi-scale depth features fused into the framework.","marker":"[24]"},{"why":"Hiera is the visual backbone architecture whose parameters are initialized with SAM2 weights.","marker":"[19]"},{"why":"DINOv2-Large is the backbone of the geometric encoder inside Depth Anything V2.","marker":"[13]"},{"why":"The DPT decoder produces the multi-scale depth embeddings used by the geometric encoder.","marker":"[17]"},{"why":"EPIC-KITCHENS VISOR defines the egocentric segmentation task, the training and validation splits, and the test set used for the reported scores.","marker":"[5]"}],"fun_headline_variants":["Depth cues push SAM2 video segmentation to 90.1 J&F","SAM2 plus depth beats fine-tuned SAM2 on egocentric VISOR","90.1 J&F: SAM2 init and depth features lift video object segmentation","Depth Anything V2 boosts SAM2-based Cutie to 90.1 J&F"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Depth Anything V2 produces depth features that stay accurate and informative in fast-moving, heavily occluded egocentric scenes, and those features add signal beyond what SAM2-pretrained visual features already encode.","fun_headline_variants_meta":{"raw":{"variants":["Depth cues push SAM2 video segmentation to 90.1 J&F","SAM2 plus depth beats fine-tuned SAM2 on egocentric VISOR","90.1 J&F: SAM2 init and depth features lift video object segmentation","Depth Anything V2 boosts SAM2-based Cutie to 90.1 J&F"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000453,"raw_usage":{"total_tokens":2208,"prompt_tokens":804,"completion_tokens":1404,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":420,"completion_tokens_details":{"reasoning_tokens":1316}},"tokens_in":420,"tokens_out":1404,"duration_ms":10427,"temperature":1.0,"reasoning_tokens":1316,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T05:49:47.134906+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the validation ablation with the Depth Anything V2 branch fed scrambled or heavily blurred depth maps while keeping all other settings identical; if J&F stays at 88.7% or higher, the reported depth contribution comes from the fusion module's parameters rather than from geometric information.","supporting_citations":[{"cited_title":"Putting the object back into video object segmentation","cited_arxiv_id":null,"evidence_quote":"Cutie is the baseline framework that provides the long-term object-aware memory mechanism being extended."},{"cited_title":"Hi- era: A hierarchical vision transformer without the bells-and- whistles","cited_arxiv_id":null,"evidence_quote":"Hiera is the visual backbone architecture whose parameters are initialized with SAM2 weights."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"DINOv2-Large is the backbone of the geometric encoder inside Depth Anything V2."},{"cited_title":"Vi- sion transformers for dense prediction","cited_arxiv_id":null,"evidence_quote":"The DPT decoder produces the multi-scale depth embeddings used by the geometric encoder."},{"cited_title":"Epic-kitchens visor benchmark: Video segmenta- tions and object relations","cited_arxiv_id":null,"evidence_quote":"EPIC-KITCHENS VISOR defines the egocentric segmentation task, the training and validation splits, and the test set used for the reported scores."}],"review_version":1}