{"id":"5139a9d5-20ee-4ca0-9cf3-7493275223bd","arxiv_id":"2412.13803","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"M3-VOS is a new video object segmentation benchmark focused on phase transitions, and the ReVOS reverse-propagation module improves accuracy on it.","lead":"This paper introduces a new video segmentation benchmark, M3-VOS, with 479 clips of objects changing phase, such as melting, splashing, and dissolving. It also proposes ReVOS, a plug-and-play module that segments videos backward as well as forward, improving accuracy on this benchmark while staying competitive on standard ones.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Training data for the fusion module is undisclosed; if M3-VOS videos were used in finetuning, the reported improvement is test-set leakage, not evidence for the plug-and-play claim.","rationale":"The most load-bearing requirement for the central claim is that the reported improvement is a genuine effect of reverse refinement rather than an artifact of the evaluation protocol. The paper omits the training data for the only component that is trained (the Readout Fusion Module). Since M3-VOS is introduced as a new benchmark with no stated train/test split, it is possible that the fusion module was optimized on the same videos used to report the improvement. This directly undermines the empirical foundation; unlike the entropy contradiction on DAVIS, which only weakens the paper's explanatory narrative, training leakage would invalidate the evidence itself. The reader's weakest assumption about the entropy trend is real but secondary: the method's success does not logically require the entropy hypothesis to be correct, as long as reverse propagation provides a useful refinement signal. The training-data disclosure, by contrast, is a necessary condition for interpreting the numbers in Table 4. Therefore the reader's conditional verdict is appropriate, but the condition must include explicit disclosure of the training set and a re-evaluation on strictly disjoint data. No ad hominem is intended; this is an addressable reporting gap.","tokens_in":16813,"tokens_out":10742,"duration_ms":91707,"concrete_test":"Request the precise training set used for the Readout Fusion Module, including any M3-VOS splits. Then retrain the fusion module using only the same public training splits used by Cutie (e.g., YouTubeVOS and DAVIS training sets) and evaluate on M3-VOS full/core with no video overlap. If the improvement over Cutie shrinks to the level of the DAVIS/YouTubeVOS deltas (≤0.4 J) or is not reproducible across seeds, the general plug-and-play claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 5.1 states that 'we froze all the parameters of Cutie and only trained the Readout Fusion Module' but never identifies the training data. The paper's central empirical evidence is the improvement of ReVOS over Cutie on M3-VOS full (+1.0 J) and core (+1.7 J); on DAVIS and YouTubeVOS the differences are within 0.4 points and inconsistent. M3-VOS full and core are defined as evaluation subsets, and no train/validation split for M3-VOS is described anywhere; the ablation 'M3-VOS mid set' (§6.4) is also undefined. If the Readout Fusion Module was finetuned on any M3-VOS videos that appear in the evaluation splits, the comparison with the frozen Cutie backbone is not a test of generalization but of memorization. Because the only substantial gains are on M3-VOS, this omission is load-bearing: without knowing the exact training set, the central claim of a general post-hoc improvement cannot be evaluated.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces M3-VOS, a new benchmark of 479 high-resolution videos for video object segmentation of objects undergoing intra-phase and cross-phase transitions, with dense instance masks at 30 fps, bilingual text descriptions, and a taxonomy of phases and transitions. The authors evaluate several state-of-the-art VOS methods on this benchmark and find that appearance-based propagation models degrade on phase-transition videos. They propose ReVOS, a plug-and-play reverse-propagation module that runs a backward pass over a sliding window, boosts the final mask, and fuses forward and backward readout features, using a frozen Cutie backbone and a trained Readout Fusion Module. Their main empirical claim is that ReVOS improves Cutie on M3-VOS full (+1.0 J) and M3-VOS core (+1.7 J) while remaining comparable on DAVIS and YouTubeVOS, and that this improvement is explained by an entropy-increasing forward process and an entropy-reducing reverse process.","tokens_in":17081,"tokens_out":2668,"duration_ms":26940,"significance":"If the benchmark and the ReVOS method hold up, this is a useful contribution: the dataset targets an underexplored axis of object change (phase transitions), with a relatively large number of videos, dense annotation, multiple scenarios, and explicit quality-control procedures including dual-model cross-validation. The ReVOS framework is attractive as a post-hoc refinement that does not require retraining the backbone, and the paper includes both a motivation from disorder analysis and ablations of its components. However, the empirical evidence is currently not conclusive because the training data for the fusion module is undisclosed, the entropy hypothesis is contradicted by one of the four datasets in Table 3, and the reported gains are small and reported without error bars. The dataset and code are promised but not yet available, which limits verification of the annotation-quality claims.","major_comments":[{"comment":"Section 5.1 states that 'we froze all the parameters of Cutie and only trained the Readout Fusion Module' but never specifies which videos or annotations were used for that training. If any M3-VOS videos appear in the training set, then the reported gains on M3-VOS full and core would reflect test-set leakage, not a general post-hoc improvement. Because the only substantial gains are on M3-VOS, the exact training split is load-bearing; please disclose it completely, or re-run the evaluation on a split that is provably disjoint from the finetuning data.","section":"5.1"},{"comment":"The entropy hypothesis is contradicted by the DAVIS'17 row in Table 3: the average LBP entropy h_LBP decreases from 4.72 (first half) to 4.68 (second half). The paper states that 'in most datasets' the latter half has higher h_LBP, which is true, but the method is claimed to work on DAVIS and YouTubeVOS as well. As written, the motivating narrative does not explain why reverse propagation should help on a dataset where disorder decreases. Please either report the per-video or per-dataset breakdown with a statistical test, or reformulate the hypothesis to cover cases where the monotonicity does not hold.","section":"Table 3 and Section 5"},{"comment":"The evaluation of DeAOT excludes 'a few long video cases' due to insufficient memory. Since excluding videos selectively changes the composition of the test set, the comparison in Table 4 is not uniform across methods. Please report the number of excluded videos, the criterion for exclusion, and provide an additional comparison where all methods are run on the same subset, or use a memory-bounded DeAOT configuration that can process all videos.","section":"6.1"},{"comment":"The central empirical claim is based on differences of 1.0 and 1.7 J points on M3-VOS, while DAVIS and YouTubeVOS differences are within 0.4 points and inconsistent across metrics. The paper does not report standard deviations or multiple runs. Please provide error bars, multiple seeds, or a significance test, especially because the advantage on the benchmark used for the headline claim may not be statistically distinguishable from noise.","section":"Table 4"}],"minor_comments":[{"comment":"Equation (2) does not define the normalization of H(i); if H is a histogram of frequencies, it should be normalized to sum to 1, and N should be specified (e.g., number of LBP bins).","section":"Eq. (2)"},{"comment":"The ablation study uses 'M3-VOS mid set,' which is not defined anywhere in the main text or the supplementary material; please define this evaluation subset or rename it to match existing subsets.","section":"6.4"},{"comment":"The footnotes under Table 4 contain unclear and partially garbled text ('the grep region contains 469 objects', 'Video clip of Other Objects is too long'), and the asterisk/dagger markers for DeAOT are used inconsistently. These need to be rewritten for clarity.","section":"Table 4 footnotes"},{"comment":"The sentence 'The model is trained on 4 A100 GPUs for 10 hours' is fine, but the description of the training data (e.g., whether it includes M3-VOS or only pretraining datasets) should be moved to a dataset-table format for precision.","section":"5.1"},{"comment":"Some typos remain, e.g., 'V olunteers' in Section 4.3.2 and 'inter-phase' when the paper elsewhere uses 'cross-phase'; please proofread.","section":"Section 7"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern about undisclosed training data is real and is the most important issue in this manuscript. I do not see derivation-level circularity because Cutie is frozen, but the dataset was annotated with Cutie assistance, and ReVOS is built on Cutie; moreover, the only large gains are on M3-VOS, so the missing train/test split disclosure is essential. I would ask the authors to clarify the training data for the fusion module and to provide error bars before resubmission. The benchmark itself appears useful, and the paper is publishable if these empirical gaps are closed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this paper is worth a referee's time, but the authors need to answer one question before the method claim is credible: what data was the Readout Fusion Module trained on?\n\nWhat is actually new: the phase taxonomy (solid/liquid/aerosol, intra- vs. cross-phase), the 479-video benchmark with dense 30 fps masks, bilingual text, and a core subset. This fills a genuine gap; appearance-change benchmarks like VOST and VSCOS don't cover phase transitions, and the paper shows that existing models lose 10-20 points on these videos. The annotation pipeline with the multi-level tool and the SAM2/Cutie cross-validation is described in enough detail that I believe the mask quality. The qualitative failure analysis is useful.\n\nWhat the paper does well: the dual-model cross-validation is a thoughtful way to reduce model bias in assisted annotation, and the Jcc metric is a reasonable addition. The ablations are standard, and the authors are upfront about limitations.\n\nWhere it gets soft: Section 5.1 says the Cutie backbone is frozen and only the Readout Fusion Module is trained, but it never says what videos that module is trained on. The only sizable gains are on M3-VOS full and core; on DAVIS and YouTubeVOS the differences vs. Cutie are within noise. If the fusion module saw any M3-VOS training videos that overlap the evaluation subsets, the +1.0/+1.7 J gains are not evidence of a general plug-and-play module. There's no described train/validation split for M3-VOS, and the ablation 'mid set' is undefined. That's a load-bearing omission, not a stylistic one. Separately, the entropy-increasing hypothesis is contradicted by the DAVIS numbers in Table 3, so the theoretical story doesn't clearly explain the gain. No data or code are released yet, and there are no error bars. These are fixable in a revision.\n\nThe benchmark contribution survives these concerns. I would cite it if the data is actually released. But I would not yet treat the ReVOS numbers as evidence for reverse refinement as a general technique.\n\nRecommendation: send it to peer review, but make the authors specify the training data, define the mid set, and report train/test separation for M3-VOS before acceptance.","headline":"M3-VOS is a genuinely useful benchmark, but the ReVOS gains are not interpretable until the training data for the fusion module is disclosed.","tokens_in":17545,"tokens_out":3461,"would_cite":true,"duration_ms":28842,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper introduces a phase taxonomy and a 479-video benchmark for objects undergoing phase transitions, and a reverse-refinement module that raises segmentation accuracy on it.","keywords":["video object segmentation","phase transition","benchmark","reverse propagation","mask entropy","Local Binary Pattern","semi-supervised VOS","object state change"],"falsifier":"Take a video in which a scattered pile of fragments is assembled into a single block, so the mask's $h_{\\mathrm{LBP}}$ visibly decreases over time; if ReVOS still shows the same J gain as on ordinary phase-transition videos, the entropy-reduction explanation is not what drives the improvement, and the reverse pass must be helping for a different reason (such as temporal averaging or additional refinement steps).","tokens_in":16646,"feed_emoji":"🎬","tokens_out":6379,"duration_ms":52175,"temperature":0.7,"pith_summary":"The paper introduces the concept of object phase to video object segmentation: a macroscopic taxonomy of solid, liquid, and aerosol/gas phases, each with characteristic intra-phase transitions (splitting, flowing, diffusing) and cross-phase transitions (melting, freezing, sublimating). To test how well segmentation models handle these changes, it builds M3-VOS, a benchmark of 479 high-resolution videos with dense 30 fps instance masks spanning 14 everyday scenarios. Evaluating state-of-the-art methods shows that appearance-based trackers degrade sharply on fluids, gases, and cross-phase transitions. The paper's central proposal is ReVOS, a plug-and-play module that runs mask propagation backward through the video and fuses forward and reverse readouts; on the frozen Cutie backbone it raises J on M3-VOS from 74.6 to 75.6 while staying comparable on DAVIS and YouTubeVOS.","feed_headline":"Reverse replay lifts video segmentation through phase transitions","feed_subtitle":"Running mask propagation backward lifts accuracy on phase-change videos from 74.6 to 75.6 without hurting standard benchmarks.","key_machinery":"The central mechanism is bidirectional mask propagation built on a frozen Cutie-style matching backbone. In the forward pass, the model keeps a sliding window of length T of image and memory readout features; after predicting the final frame, ReVOS re-introduces the boosted final mask as the starting point and propagates it backward through the same window, using a separate reverse memory that is cleared at the start of each backward run. The booster applies $M = \\sigma(\\alpha X_{\\mathrm{decode}})$ to the decoded logits, and the readout fusion module combines the forward readout $X_F$ and backward readout $X_B$ (via convolution, pooling, concatenation, and dot product) to decode the final mask. The motivating measurement is the Local Binary Pattern entropy $h_{\\mathrm{LBP}}$ of the object mask, which the paper reports increases from 4.28 to 4.37 (first to latter half) on M3-VOS, supporting the claim that the backward pass is entropy-reducing.","core_discovery":"The core claim is that the forward process of mask propagation is entropy-increasing: as objects undergo phase transitions, their binary masks become more disordered, which is measured by the Local Binary Pattern entropy $h_{\\mathrm{LBP}}$ of the mask. Consequently, a reverse pass that begins from the well-annotated final mask and propagates backward traverses a gradually simplifying sequence, and fusing that reverse readout with the forward readout yields better segmentation. ReVOS realizes this with three additions on top of a frozen propagation backbone: a reverse memory that stores high-resolution features for the backward sliding window, a booster that inflates the forward mask so small object parts are not lost, and a readout fusion module that combines forward and backward features before decoding. The paper reports that this raises J on M3-VOS full from 74.6 (Cutie) to 75.6, with gains across all four phase-transition categories and the largest gain on cross-phase transitions, while remaining comparable on DAVIS'17 and YouTubeVOS.","pith_inferences":["The entropy measure is computed on the binary mask, not the video pixels; one could test whether ReVOS's gain disappears on artificially constructed videos with decreasing mask entropy, which would separate the entropy mechanism from generic temporal smoothing.","The annotation pipeline's color-difference masking, designed for transparent smoke and splashes, could be reused to label amorphous objects in existing segmentation datasets, effectively extending benchmarks without new capture.","If reverse propagation helps whenever the target's disorder grows, the same readout-fusion idea may transfer to other forward-in-time prediction problems, such as optical flow or video prediction, where a backward pass from a known final state could refine estimates.","Since only the readout fusion module is trained while the backbone stays frozen, a natural extension is joint training of the backbone with both passes; the paper reports only frozen-backbone results, so the headroom of a fully tuned bidirectional model remains untested."],"forward_implications":["ReVOS improves J on M3-VOS full from 74.6 to 75.6 and on M3-VOS core from 64.6 to 66.3, using the frozen Cutie backbone, so reverse refinement is a post-hoc plug-in rather than a retrained model.","The gain appears across all four phase-transition groups (intra-solid, intra-liquid, intra-gas, cross-phase), with the largest margins on cross-phase transitions, so the method specifically addresses the benchmark's hardest cases.","On standard benchmarks, ReVOS stays comparable to Cutie: DAVIS'17 val J 86.0 vs 85.6 and YouTubeVOS overall G 86.8 vs 86.8, so the plug-in does not sacrifice established VOS performance.","A larger reverse interval L (up to 60) improves both performance and FPS, while a longer sliding window T trades FPS for accuracy, giving users a practical speed-quality knob."],"supporting_citations":[{"why":"Frozen mask-propagation backbone that ReVOS wraps; also supplies the interactive annotation backend and the Cutie baseline.","marker":"[4]"},{"why":"Memory-based VOS baseline and source of the working-memory concept that ReVOS reuses as reverse memory.","marker":"[2]"},{"why":"Supplies Local Binary Patterns, the texture descriptor whose histogram defines mask entropy $h_{\\mathrm{LBP}}$.","marker":"[20]"},{"why":"Strong baseline and second model used in the dual-model cross-validation of annotations.","marker":"[25]"},{"why":"Prior benchmark for objects that change appearance; source of the $J_{tr}$ metric and of the void-mask annotation practice.","marker":"[29]"},{"why":"Benchmark for state-changing objects; source of the connected-component Jaccard metric $J_{cc}$ and comparison in the main tables.","marker":"[34]"},{"why":"Standard DAVIS'17 validation set used for comparing ReVOS against Cutie and other baselines.","marker":"[23]"},{"why":"YouTubeVOS validation benchmark used for the overall G/J/F comparison.","marker":"[30]"},{"why":"DeAOT, a strong memory-based baseline included in all main comparisons.","marker":"[33]"},{"why":"RMem, a restricted-memory-bank baseline included in the main comparisons.","marker":"[35]"}],"fun_headline_variants":["Backward replay sharpens phase-change video segmentation","Reverse pass lifts segmentation on phase-transition videos","Fusing forward and reverse masks boosts video object segmentation","Phase-transition videos see gains from backward refinement","Reverse refinement improves segmentation across phase changes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that an object's mask becomes progressively more disordered over the video, so every backward pass runs from a low-entropy final frame into a simpler sequence; the paper's own table shows that this trend reverses on DAVIS'17 ($h_{\\mathrm{LBP}}$ falls from 4.72 to 4.68), so the entropy story does not explain the improvement on every video.","fun_headline_variants_meta":{"raw":{"variants":["Backward replay sharpens phase-change video segmentation","Reverse pass lifts segmentation on phase-transition videos","Fusing forward and reverse masks boosts video object segmentation","Phase-transition videos see gains from backward refinement","Reverse refinement improves segmentation across phase changes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000171,"raw_usage":{"total_tokens":1304,"prompt_tokens":1007,"completion_tokens":297,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":623,"completion_tokens_details":{"reasoning_tokens":227}},"tokens_in":623,"tokens_out":297,"duration_ms":3340,"temperature":1.0,"reasoning_tokens":227,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T12:46:18.837298+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a video in which a scattered pile of fragments is assembled into a single block, so the mask's $h_{\\mathrm{LBP}}$ visibly decreases over time; if ReVOS still shows the same J gain as on ordinary phase-transition videos, the entropy-reduction explanation is not what drives the improvement, and the reverse pass must be helping for a different reason (such as temporal averaging or additional refinement steps).","supporting_citations":[{"cited_title":"Putting the object back into video object segmentation","cited_arxiv_id":null,"evidence_quote":"Frozen mask-propagation backbone that ReVOS wraps; also supplies the interactive annotation backend and the Cutie baseline."},{"cited_title":"Xmem: Long- term video object segmentation with an atkinson-shiffrin memory model","cited_arxiv_id":null,"evidence_quote":"Memory-based VOS baseline and source of the working-memory concept that ReVOS reuses as reverse memory."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies Local Binary Patterns, the texture descriptor whose histogram defines mask entropy $h_{\\mathrm{LBP}}$."},{"cited_title":"Breaking the” object” in video object segmentation","cited_arxiv_id":null,"evidence_quote":"Prior benchmark for objects that change appearance; source of the $J_{tr}$ metric and of the void-mask annotation practice."},{"cited_title":"Video state-changing object segmentation","cited_arxiv_id":null,"evidence_quote":"Benchmark for state-changing objects; source of the connected-component Jaccard metric $J_{cc}$ and comparison in the main tables."},{"cited_title":"Youtube-vos: Sequence-to-sequence video object segmentation, 2018","cited_arxiv_id":null,"evidence_quote":"YouTubeVOS validation benchmark used for the overall G/J/F comparison."},{"cited_title":"Decoupling features in hierar- chical propagation for video object segmentation.Advances in Neural Information Processing Systems, 35:36324–36336,","cited_arxiv_id":null,"evidence_quote":"DeAOT, a strong memory-based baseline included in all main comparisons."},{"cited_title":"Rmem: Re- stricted memory banks improve video object segmentation","cited_arxiv_id":null,"evidence_quote":"RMem, a restricted-memory-bank baseline included in the main comparisons."}],"review_version":1}