{"id":"62c6b2d8-2bf5-4fbc-bb7a-f0f2dbd794df","arxiv_id":"2605.18013","paper_version":1,"verdict":"CONDITIONAL","confidence":"LOW","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"TinySAM 2 reaches 90% of SAM 2.1 performance on DAVIS and SA-V using 7% of the memory tokens and 3% of the training data via frame selection, spatial average pooling, temporal similarity-based token pruning, and a RepViT image encoder.","lead":"TinySAM 2 introduces memory quality management and joint spatial-temporal token compression to shrink SAM 2's memory and compute needs while keeping most of its video segmentation accuracy. A smart generalist might read it to see how large foundation models can be made practical for phones and edge devices.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Average pooling in spatial compression may discard boundary-critical features needed for precise long-term segmentation","rationale":"The reader's weakest assumption directly identifies the information-preservation risk in the compression pipeline; my analysis refines it to the specific mechanism (average pooling) and proposes a minimal ablation that isolates its effect without requiring new training.","tokens_in":1774,"tokens_out":300,"duration_ms":15758,"concrete_test":"Re-run the DAVIS 2017 validation set using the authors' released checkpoint but with spatial pooling disabled (replace with identity or strided sampling) while keeping temporal selection; if the J&F score rises by >4 points relative to the reported TinySAM 2 result, the pooling step is the dominant source of the remaining 10% gap.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline 90% performance claim with 7% memory tokens rests on the joint-spatial-temporal compression preserving all task-relevant information. Average pooling over spatial tokens (methods section on joint compression) inherently smooths high-frequency details such as object edges and fine textures; when combined with similarity-based temporal selection, tokens that are locally dissimilar yet globally important for re-identification after occlusion can be dropped. This assumption is least secure for long video sequences where memory-bank accumulation amplifies any per-step loss, yet no per-component ablation isolates the contribution of pooling versus selection on boundary metrics (J&F on DAVIS).","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper proposes TinySAM 2, a compressed version of SAM 2 for video object segmentation and tracking. It adds a memory quality management mechanism to retain high-informative historical frames, introduces joint spatial-temporal token compression (average pooling in the spatial domain followed by similarity-based selection across the memory bank in the temporal domain), and substitutes RepViT for the image encoder. Experiments on DAVIS and SA-V are reported to show that TinySAM 2 retains 90% of SAM 2.1 performance while using only 7% memory tokens and 3% training data.","tokens_in":1913,"tokens_out":502,"duration_ms":27396,"significance":"If the performance retention claims can be substantiated through detailed ablations and statistical reporting, the work would offer a practical route to deploying video foundation models on memory-constrained devices. The deterministic compression approach avoids the need for full retraining and could generalize to other memory-bank-based video models.","major_comments":[{"comment":"Abstract and Experiments section: The headline claim of 90% performance retention with 7% memory tokens and 3% training data is presented without error bars, standard deviations across runs, or explicit description of the measurement protocol (including whether post-hoc selection of high-informative frames was applied uniformly). This makes it impossible to judge the robustness of the central efficiency-performance tradeoff.","section":"Abstract / Experiments"},{"comment":"Methods paragraph on joint-spatial-temporal compression: Average pooling is used to compress spatial tokens before temporal similarity selection, yet no component-wise ablation isolates the pooling step's effect on boundary-sensitive metrics such as J&F on DAVIS. Because pooling inherently smooths high-frequency edge information, this choice risks systematic degradation in long-term re-identification after occlusion; the absence of such an ablation leaves the information-preservation assumption untested.","section":"Methods (joint-spatial-temporal compression)"}],"minor_comments":[{"comment":"The abstract should explicitly state the SAM 2.1 baseline version and the precise definition of 'memory tokens' used to compute the 7% figure.","section":"Abstract"},{"comment":"Add a short related-work paragraph contrasting the proposed deterministic selection with learned memory compression methods in prior video segmentation literature.","section":null}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive comments, which help strengthen the presentation of our efficiency claims. We address each major comment below and have revised the manuscript to incorporate additional statistical reporting and ablations.","responses":[{"response":"We agree that error bars and a precise protocol description are necessary to substantiate the central claims. In the revised manuscript we now report standard deviations over three independent runs with different random seeds for the key J&F and mIoU metrics on both DAVIS and SA-V. We have also expanded the Experiments section to explicitly state that frame selection follows the deterministic memory quality management mechanism uniformly across all sequences, with no post-hoc selection of high-informative frames. These additions allow readers to assess the robustness of the reported 90 % retention figure.","revision_made":"yes","referee_comment":"[Abstract / Experiments] Abstract and Experiments section: The headline claim of 90% performance retention with 7% memory tokens and 3% training data is presented without error bars, standard deviations across runs, or explicit description of the measurement protocol (including whether post-hoc selection of high-informative frames was applied uniformly). This makes it impossible to judge the robustness of the central efficiency-performance tradeoff."},{"response":"We acknowledge the value of isolating the spatial pooling component. Although the original submission contained an ablation of the combined spatial-temporal compression, it did not separate the pooling step. In the revised manuscript we have added a dedicated component-wise ablation that measures J&F on DAVIS with and without spatial average pooling (while keeping temporal similarity selection fixed). The results show only a small degradation, indicating that the subsequent temporal selection largely preserves the information needed for re-identification after occlusion. A short discussion of this finding has been inserted in the Methods section.","revision_made":"yes","referee_comment":"[Methods (joint-spatial-temporal compression)] Methods paragraph on joint-spatial-temporal compression: Average pooling is used to compress spatial tokens before temporal similarity selection, yet no component-wise ablation isolates the pooling step's effect on boundary-sensitive metrics such as J&F on DAVIS. Because pooling inherently smooths high-frequency edge information, this choice risks systematic degradation in long-term re-identification after occlusion; the absence of such an ablation leaves the information-preservation assumption untested."}],"tokens_in":1480,"tokens_out":495,"duration_ms":46802,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"TinySAM 2 reaches 90 percent of SAM 2.1 performance on video segmentation while dropping to 7 percent memory tokens and 3 percent training data. That is the central result worth noting if you work on deploying these models anywhere with tight hardware limits.","headline":"TinySAM 2 shows a practical compression recipe for SAM 2's memory bank but the experiments are too light on details to fully trust the 90 percent performance claim.","tokens_in":2418,"tokens_out":133,"would_cite":false,"duration_ms":37191,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":{"model":"grok-4.3","evidence":[{"relation":"unclear","rs_module":"IndisputableMonolith/Cost/FunctionalEquation.lean","rs_theorem":"washburn_uniqueness_aczel","paper_passage":"average pooling is employed to first compress redundancy tokens in the spatial domain... informative tokens are selected across frames in the memory bank based on token-level similarity measurement"}],"headline":"TinySAM 2 token-compression pipeline lies outside RS forcing chain","alignment":"orthogonal","rationale":"The paper's central machinery (average pooling over spatial tokens + cosine-similarity temporal selection inside a memory bank) is a standard engineering compression technique for video segmentation. It has no structural overlap with the RS recognition-cost J(x), φ-ladder, 8-tick periodicity, or the distinction-to-spacetime forcing theorems. The domain (cs.CV model efficiency) is one on which the RS framework explicitly has no opinion.","tokens_in":52375,"confidence":"high","tokens_out":217,"duration_ms":9986,"cache_read_input_tokens":32896,"cache_creation_input_tokens":0},"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"TinySAM 2 reduces SAM 2 memory tokens to 7 percent while preserving 90 percent of video segmentation performance.","keywords":["video segmentation","memory compression","token selection","lightweight model","object tracking","SAM 2"],"falsifier":"A clear accuracy drop on video clips that contain fast motion or long occlusions would show the compression has lost critical information.","tokens_in":2669,"feed_emoji":"📹","tokens_out":587,"duration_ms":26977,"temperature":0.7,"pith_summary":"The paper presents TinySAM 2 as a compact video segmentation model built on SAM 2. It adds a memory quality management step to keep only high-value past frames and applies joint spatial-temporal compression to shrink the stored tokens. Average pooling first removes spatial redundancy within frames, after which similarity measurements pick the most useful tokens across the memory bank. A RepViT backbone further cuts parameter count. If these steps hold, the model supports accurate tracking and segmentation on devices that cannot run the full SAM 2.","feed_headline":"TinySAM 2 cuts SAM 2 memory tokens to 7 percent at 90 percent accuracy","feed_subtitle":"Memory quality checks and spatial-temporal compression deliver efficient video tracking with far less storage and training data.","key_machinery":"Joint-spatial-temporal token compression, which first applies average pooling to reduce spatial tokens inside each frame and then selects informative tokens across memory-bank frames by token-level similarity.","core_discovery":"TinySAM 2 achieves 90 percent of SAM 2.1 performance on DAVIS and SA-V benchmarks using only 7 percent of the memory tokens and 3 percent of the training data. The gains come from a memory quality management mechanism that retains informative historical frames and a joint-spatial-temporal token compression method that applies average pooling in the spatial domain followed by similarity-based selection in the temporal domain across the memory bank. The image encoder is replaced with RepViT to lower overall model size.","pith_inferences":["Adaptive selection thresholds might further improve results on videos of varying complexity.","The technique could combine with other lightweight encoders to push efficiency gains higher.","Real-time applications on mobile devices become more feasible once memory footprint falls this far."],"forward_implications":["The model runs with far lower memory and compute, opening video segmentation to hardware with limited capacity.","Experiments confirm the approach works on standard benchmarks while cutting storage and training costs.","The same compression pattern can be tested on other memory-dependent video models."],"fun_headline_variants":["TinySAM 2 achieves 90 percent SAM 2.1 performance with 7 percent memory tokens","TinySAM 2 matches 90 percent SAM 2.1 results at 7 percent memory tokens","TinySAM 2 reaches 90 percent SAM 2.1 performance using 7 percent memory tokens","TinySAM 2 uses 7 percent memory tokens for 90 percent SAM 2.1 performance"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"Average pooling plus similarity-based selection across the memory bank keeps every detail required for correct long-term tracking even when objects move quickly or become hidden.","fun_headline_variants_meta":{"raw":{"variants":["TinySAM 2 achieves 90 percent SAM 2.1 performance with 7 percent memory tokens","TinySAM 2 matches 90 percent SAM 2.1 results at 7 percent memory tokens","TinySAM 2 reaches 90 percent SAM 2.1 performance using 7 percent memory tokens","TinySAM 2 uses 7 percent memory tokens for 90 percent SAM 2.1 performance"]},"model":"grok-4.3","cost_usd":0.008904,"raw_usage":{"total_tokens":4041,"prompt_tokens":744,"num_sources_used":0,"completion_tokens":101,"cost_in_usd_ticks":89037000,"prompt_tokens_details":{"text_tokens":744,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":3196,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":744,"tokens_out":101,"duration_ms":33688,"temperature":1.0,"reasoning_tokens":3196,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-05-20T12:11:37.871413+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A clear accuracy drop on video clips that contain fast motion or long occlusions would show the compression has lost critical information.","supporting_citations":[],"review_version":1}