{"id":"d0df2a97-f326-451c-8edb-5d77433bdd1c","arxiv_id":"2608.03912","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"StreamDAM makes a state-of-the-art video object segmentation tracker run in real time by using a learned presence signal to control its memory, recovering almost all offline accuracy and exceeding the offline model on a curated hard benchmark.","lead":"A new video-tracking system, StreamDAM, rebuilds the memory pipeline of a top-tier video object segmentation model so it can run at 30 frames per second without losing accuracy. On the hardest test content it even beats the offline model it is built from, which means real-time deployments could use quality-tier trackers rather than distilled ones.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Presence-head gains are shown only on MOSE-curated panels; without a streamed ablation on standard benchmarks, the 'exceeds offline on hardest content' claim lacks cross-dataset support.","rationale":"The reader's weakest assumption is that the learned presence head generalizes to unseen benchmarks. Our concern is more specific: the reported benefit of the presence head is concentrated on MOSE-hard, which is curated from the same distribution as the head's training data, and the attribution suggests no benefit on DAVIS and a small cost on VOST. This does not contradict the reader's conditional verdict; rather, it sharpens one of the conditions the reader already imposes: validating the hard-content claim on a standard, independently defined panel. If the proposed ablation fails, the central claim would be rejected, but that is an unresolved empirical condition, so the manuscript remains correctly classified as CONDITIONAL pending the requested evidence. We therefore recommend no change to the reader's verdict.","tokens_in":7608,"tokens_out":6685,"duration_ms":62912,"concrete_test":"Add a streamed ablation row to Table II: 'StreamDAM without presence head' (using the best fixed policy found on MOSE-train), reported per benchmark with seed error bars. Also re-run the hard-content claim on an independently defined hard panel, such as DAVIS-2017 validation sequences containing at least one disappearance event or a held-out LVOS split. If the presence head's advantage over the fixed policy disappears or drops below noise on these out-of-distribution sets, the central 'presence-aware' claim and the 'exceeds offline on hardest content' result would not be supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of a learned presence signal that recovers the offline model's accuracy on hard content is supported primarily by the MOSE-hard panel (Table II), which is curated from the same MOSE-train distribution used to train the presence head (Sec. III-D, App. B). Fig. 1's oracle-gap comparison is also on held-out MOSE-train, so it cannot establish cross-dataset generalization. The paper's own attribution paragraph says the DAVIS gain comes from the uniform mask-initialization fix, not the presence governor, and that the presence governor carries the rest of MOSE-hard 'at a small cost on VOST'. No per-benchmark streamed ablation with a fixed policy is reported, so the only evidence that the learned presence signal beats a fixed policy on out-of-distribution benchmarks is missing. If the presence head only helps in-distribution content, the headline 'on the hardest content exceeds the offline model' would be an artifact of panel curation rather than a general property, and the 'presence-aware' contribution would reduce to a speed optimization plus a dataset-specific governor.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"StreamDAM modifies the DAM4SAM-L video object segmentation model so that its memory pipeline can run under a 30 fps streaming clock. The paper formalizes a zero-order-hold streaming protocol, identifies the memory pipeline as the source of both the latency failure and the absence-blindness failure, and then introduces three in-model optimizations (graph-captured backbone, host-synchronization removal, resolution-capped distractor introspection) together with a learned GRU-based presence head that controls memory admission, recency stride, output suppression, and re-detection. The evaluation on DAVIS-2017 val, LVOS val, VOST val, and a curated MOSE-hard panel against five baselines reports the best mean streamed J&F (0.727), recovering about 97% of the offline model's streamed gap, and exceeding the offline model on MOSE-hard (0.626 streamed vs. 0.610 offline). The manuscript also includes a negative-results ledger and an explicit limitations section.","tokens_in":7883,"tokens_out":5857,"duration_ms":52524,"significance":"If the central claims hold, the paper makes a useful and practical contribution: it localizes the streaming failure of a quality-tier tracker to the memory pipeline, shows that in-model optimizations can bring the pipeline under the clock while preserving outputs on most sequences, and proposes a single learned presence signal that couples four memory decisions. The experimental protocol is unusually well specified, with a timed harness, three seeds, honest bounds, and a negative-results appendix, and the limitations are stated rather than hidden. However, the headline 'exceeds the offline model on hardest content' is currently supported only by a curated subset of MOSE-train, and the lack of a streamed ablation with the presence governor disabled on the standard benchmarks leaves the generality of the learned signal unestablished. These issues are substantial but addressable with additional experiments.","major_comments":[{"comment":"The central generalization claim is not yet supported by a cross-dataset streamed ablation. The presence head is trained on a disjoint subset of MOSE-train, and MOSE-hard is curated from the same held-out MOSE-train split that is disjoint from the presence-head training corpus but drawn from the same distribution. Thus the MOSE-hard result is in-distribution for the learned signal. The attribution paragraph says the DAVIS gain comes from the uniform mask-initialization fix and that the presence governor costs a small amount on VOST, but no per-benchmark streamed comparison of StreamDAM with and without the presence governor (or against a fixed policy with thresholds tuned on the same training data) is reported. Please add a streamed ablation on DAVIS, LVOS, VOST, and MOSE-hard in which the learned presence governor is replaced by a fixed policy, so the reader can see whether the learned signal helps outside the curated panel.","section":"Sec. III-D/III-E, Table II, App. B"},{"comment":"The claim that all optimizations are 'bit-identical by construction' is contradicted by App. C, which states that the resolution cap hires_cap=512 is 'bit-identical on 5/6 seqs', and by the paper's own limitation statement about one oversize VOST sequence. This matters because the contributions section says 'emitted masks identical to the offline tracker' and the main result interprets the MOSE-hard gain as the presence governor improving the model itself. Please specify exactly which sequence differs, by how much in J&F or mask agreement, and whether any reported metric changes if that sequence is excluded; otherwise the caption should be qualified to remove 'ALL'.","section":"Table I caption and App. C"},{"comment":"No uncertainty is reported for the baselines. StreamDAM is the mean over three fresh-compile seeds with spread below one point except on LVOS, but all other methods appear as single numbers without variance. With mean margins over the nearest baseline as small as 0.012 on MOSE-hard and 0.001 on VOST, the claim of being the strongest streaming tracker needs a paired per-sequence analysis or per-benchmark standard deviations. Please report per-benchmark mean plus/minus standard deviation for all methods, or provide a paired test over sequences.","section":"Table II"},{"comment":"The mechanism analysis that 'no fixed policy can win' compares against a small set of hand-specified fixed policies and reports gap-closed percentages without variance or sample counts. Since the presence head's operating thresholds (tau, theta, p_hi, p_lo, R, K) are all tuned on training data, the comparison is not against a fair tuned fixed baseline. Please report, on the same held-out MOSE-train oracle panel, the gap-closed performance of a fixed policy with each threshold optimized on the training split, together with a bootstrap confidence interval for the 36% gap-closed number attributed to StreamDAM.","section":"Sec. III-B and Fig. 1"}],"minor_comments":[{"comment":"The scalar feature vector u_t used by the presence head is not enumerated; for reproducibility, please list the exact features, their dimensions, and how the absence run-length is computed.","section":"Sec. III-D"},{"comment":"The statement that fast baselines are 'verified with a zero hold-fraction on every spot-check' should be replaced by reporting the measured hold-fraction for every baseline on every benchmark, since spot-checks are not a complete audit.","section":"Sec. IV"},{"comment":"The ledger uses internal identifiers such as 'eeaaa67e' and 'matched-fp' without definitions; expanding these abbreviations and identifying the failing sequence class in words would make the negative results easier to interpret.","section":"App. A and App. D"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is honest about its limitations and the empirical protocol is well designed, but the headline claim depends on a curated in-distribution panel and the missing streamed ablation is the decisive issue. The bit-identical overclaim should also be fixed before publication. I did not find evidence of citation problems or scope mismatch."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Punchline: this is a solid systems-plus-mechanism paper that deserves peer review, but its flashiest claim—beating the offline model on the hardest content—rests on a curated panel drawn from the same data distribution that trained the learned controller. The authors know this and say so, which is to their credit.\n\nThe new bit is real. They isolate both streaming failures of DAM4SAM-L to the memory pipeline and fix it inside the model: CUDA-graph capture, removal of host syncs, and a resolution cap on distractor introspection get streamed compute under budget, with masks bit-identical on 5/6 sequences. That caveat matters a little, but the optimization story is persuasive. The conceptual contribution is a single learned presence signal—a tiny GRU over runtime scalars—that controls memory admission, recency stride, output suppression, and re-detection. The mechanism analysis (Fig. 1) is the strongest part: it shows why no fixed policy can simultaneously handle true disappearances and low-confidence frames. The negative-results ledger (App. A) is unusually honest and useful.\n\nWhere it gets soft. The main quantitative headline, 'exceeds the offline model on hardest content,' only holds on MOSE-hard, which is curated from held-out MOSE-train—the same distribution the presence head was trained on. The attribution paragraph concedes that on VOST the governor costs a little, and that DAVIS gains come from the mask-init fix. There is no per-benchmark streamed ablation on standard benchmarks comparing the learned governor against a fixed policy. So the generalizable value of the presence signal is not yet established. Second, no error bars for baselines; the mean gain over EdgeTAM is 0.727 vs 0.704, probably real, but a confidence interval would help. Third, code, MOSE-hard sequence list, and pre-registration details are not released, so the central result is not externally reproducible. These are fixable gaps, not fatal flaws.\n\nWho this is for: anyone working on streaming perception or real-time VOS. Even if the presence-governor idea fails to transfer, the in-model optimization and the negative results are worth reading. My recommendation: send it to peer review. Ask for the MOSE-hard list, code, and a fixed-policy ablation on the standard benchmarks. If those come back and the claim shrinks to 'on MOSE-like hard content, presence control helps,' that's still a decent paper; the current framing oversells it slightly.","headline":"Useful real-time VOS systems work with an honest but non-standard 'hardest content' claim that needs a cross-dataset ablation before it carries weight.","tokens_in":8377,"tokens_out":2689,"would_cite":false,"duration_ms":22804,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Streaming video object segmentation fails because of its memory pipeline, not its accuracy, and one learned presence signal can fix it.","keywords":["video object segmentation","streaming perception","presence estimation","memory networks","real-time tracking","zero-order hold","distractor-aware memory","SAM2"],"falsifier":"Re-run the streamed Table II evaluation with the presence head replaced by a fixed policy that always keeps the recency window dense, sets the admission threshold to $\\tau=0.34$, and disables output suppression and re-detection. If the streamed mean on MOSE-hard does not drop below 0.626 by more than the seed spread (about 0.01), the per-frame presence mechanism is not load-bearing. A second check: measure the presence head's false-absent rate on LVOS at matched hallucination; the logit baseline is 0.099, and if the head's rate exceeds that on LVOS, the claimed head benefit is contradicted.","tokens_in":1796,"feed_emoji":"🎬","tokens_out":2431,"duration_ms":66478,"temperature":0.7,"pith_summary":"The paper claims that quality-tier video object segmentation fails under a real-time 30 fps clock because the tracker's memory pipeline is both too slow and blind to whether the object is present. StreamDAM rebuilds that memory pipeline so it runs at frame rate with masks identical to the offline model, and adds one small learned presence signal that decides what enters memory, how far back to read, when to suppress output, and when to re-detect. Under the zero-order-hold streaming protocol, StreamDAM reaches a mean J&F of 0.727, recovering about 97% of the streaming gap of its offline base DAM4SAM-L (offline 0.733, naive-streamed 0.546). On the hardest curated content, MOSE-hard, the streamed score exceeds the offline model, showing the fix improves the model itself, not just how it is served.","feed_headline":"Real-time video tracker recovers 97% of offline accuracy","feed_subtitle":"One learned presence signal runs the memory at 30 fps and beats the offline model on hard clips.","key_machinery":"The load-bearing object is the coupled recency control inside the memory read $$M_t = \\mathcal{C} \\cup \\{e_j \\in B_t : |m_j| > 0 \\land j \\in W_r(t)\\},$$ where the presence filter $|m_j| > 0$ and the temporal stride $r$ together decide which past entries the tracker attends to. StreamDAM replaces this fixed presence filter with a learned presence probability $p_t$ output by a small gated recurrent unit that reads runtime scalars already computed by the tracker (logits, mask-overlap statistics, area ratios, absence run-length). That $p_t$ drives four consumers: admission into the memory bank ($j \\in M_t$ iff $p_j > \\tau$), a hysteresis-based dense/sparse recency window that goes dense for a burst after reappearance, output suppression (empty mask when $p_t < \\theta$), and re-detection after $K$ suppressed frames. The in-model speedups (graph capture of the backbone, removal of host synchronization via GPU-resident masks and byte-packed transfers, and a capped distractor introspection whose add-decision is scale-invariant) make the pipeline meet the 33 ms budget while keeping masks identical to the offline model.","core_discovery":"The central claim is that the streaming cliff in VOS is caused by the memory pipeline, not by model quality, and that a single learned presence probability $p_t$ can govern the four memory decisions (admission, recency stride, output suppression, and re-detection) so that the same DAM4SAM-L checkpoint becomes real-time and more accurate on hard content. StreamDAM achieves streamed mean J&F 0.727 across DAVIS, LVOS, VOST, and MOSE-hard, within 0.006 of the offline model, while naive stale-reuse streaming of the same base collapses to 0.546. On MOSE-hard the streamed score 0.626 exceeds the offline 0.610, because the presence governor changes the model's behavior, not just its serving. The three in-model optimizations (graph-captured backbone, host-synchronization removal, capped distractor introspection) are bit-identical by construction, so no accuracy is traded for speed.","pith_inferences":["The same presence-governed memory control could be grafted onto other memory-based trackers (e.g., SAM2, Cutie), since the control is defined at the memory-read equation level, not tied to DAM4SAM-specific components.","If the presence head is trained only on a disjoint slice of MOSE-train, deployment on benchmarks with very different absence statistics may require recalibration of the thresholds tau, theta, and K, which the paper sets on training data alone.","The paper's residual failure class, moved by neither presence consumer, suggests a second orthogonal signal is needed; the authors name a learned regime selector over memory composition as the natural next step, which could be tested by adding a memory-composition feature to the GRU input.","Because the streaming protocol is frozen and the harness is wall-clock, the reported 0.727 mean is a direct target for future streaming VOS work; a replication on the released harness could confirm the result."],"forward_implications":["StreamDAM's streamed mean J&F (0.727) beats the strongest efficiency-first baseline EdgeTAM (0.704) by 2.3 points and comes within 0.006 of the offline DAM4SAM-L.","On MOSE-hard, the streamed score (0.626) surpasses the offline model (0.610), showing that presence-governed memory improves content with disappearances and distractors.","The three in-model optimizations are bit-identical by construction, so accuracy gains are attributable to the presence governor, not to the speedups.","The presence governor's benefit is pre-registered across a cross-domain check: no family regresses beyond noise, meaning the MOSE-hard gain does not come at a systemic cost on the other benchmarks.","The re-detection mechanism (K=5) breaks self-reinforcing absent latches that otherwise produce 235-frame dead runs in memory."],"supporting_citations":[{"why":"Supplies the promptable SAM2 backbone and memory bank that DAM4SAM extends.","marker":"[1]"},{"why":"The distractor-aware base tracker (DAM4SAM) that StreamDAM streams and governs.","marker":"[2]"},{"why":"Defines the real-time stale-reuse protocol (zero-order hold) used for streamed scoring.","marker":"[4]"},{"why":"Extends the protocol to VOTS segmentation scoring, under which the streamed comparison is run.","marker":"[5]"},{"why":"EdgeTAM, the strongest efficiency-first streaming baseline StreamDAM must beat.","marker":"[6]"},{"why":"EfficientTAM, an efficiency baseline compared in Table II.","marker":"[7]"},{"why":"Defines streaming accuracy coupling latency and quality under a clock, motivating the protocol.","marker":"[12]"}],"fun_headline_variants":["Memory governor runs video tracker at 30fps, recovers 97% accuracy","Presence-aware memory makes real-time VOS accurate on hard clips","One learned presence signal controls VOS memory at 30fps","Streaming tracker matches offline with presence-guided memory","Real-time VOS: presence signal beats offline on hard content"],"cache_read_input_tokens":10624,"weakest_assumption_plain":"The one assumption the whole result rests on is that the small learned presence head, trained only on a disjoint slice of MOSE-train, remains accurate enough on unseen benchmarks (DAVIS, LVOS, VOST, MOSE-hard) to drive all four memory controls; if that signal stops generalizing, the reported gains collapse together.","fun_headline_variants_meta":{"raw":{"variants":["Memory governor runs video tracker at 30fps, recovers 97% accuracy","Presence-aware memory makes real-time VOS accurate on hard clips","One learned presence signal controls VOS memory at 30fps","Streaming tracker matches offline with presence-guided memory","Real-time VOS: presence signal beats offline on hard content"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000447,"raw_usage":{"total_tokens":2274,"prompt_tokens":979,"completion_tokens":1295,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":595,"completion_tokens_details":{"reasoning_tokens":1205}},"tokens_in":595,"tokens_out":1295,"duration_ms":10624,"temperature":1.0,"reasoning_tokens":1205,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:44:38.450579+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the streamed Table II evaluation with the presence head replaced by a fixed policy that always keeps the recency window dense, sets the admission threshold to $\\tau=0.34$, and disables output suppression and re-detection. If the streamed mean on MOSE-hard does not drop below 0.626 by more than the seed spread (about 0.01), the per-frame presence mechanism is not load-bearing. A second check: measure the presence head's false-absent rate on LVOS at matched hallucination; the logit baseline is 0.099, and if the head's rate exceeds that on LVOS, the claimed head benefit is contradicted.","supporting_citations":[{"cited_title":"SAM 2: Segment anything in images and videos,","cited_arxiv_id":null,"evidence_quote":"Supplies the promptable SAM2 backbone and memory bank that DAM4SAM extends."},{"cited_title":"A distractor-aware memory for visual object tracking with SAM2,","cited_arxiv_id":null,"evidence_quote":"The distractor-aware base tracker (DAM4SAM) that StreamDAM streams and governs."},{"cited_title":"The visual object tracking VOT2017 challenge results,","cited_arxiv_id":null,"evidence_quote":"Defines the real-time stale-reuse protocol (zero-order hold) used for streamed scoring."},{"cited_title":"The first visual object tracking segmentation VOTS2023 challenge results,","cited_arxiv_id":null,"evidence_quote":"Extends the protocol to VOTS segmentation scoring, under which the streamed comparison is run."},{"cited_title":"EdgeTAM: On-device track anything model,","cited_arxiv_id":null,"evidence_quote":"EdgeTAM, the strongest efficiency-first streaming baseline StreamDAM must beat."},{"cited_title":"Efficient track anything,","cited_arxiv_id":null,"evidence_quote":"EfficientTAM, an efficiency baseline compared in Table II."},{"cited_title":"Towards streaming perception,","cited_arxiv_id":null,"evidence_quote":"Defines streaming accuracy coupling latency and quality under a clock, motivating the protocol."}],"review_version":2}