{"id":"2ed9b631-8b0f-4db8-a66e-8cae5e6c98b3","arxiv_id":"2501.12060","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"GSVC represents and compresses video frames as collections of 2D Gaussian splats, using frame prediction, splat pruning and augmentation, and dynamic key-frame selection to match mainstream codec quality with fast decoding.","lead":"A research team at NUS proposes encoding video as collections of flat 2D Gaussian splats that are predicted, pruned, and augmented frame by frame. They report quality close to AV1 and HEVC at much faster decoding, with caveats about exactly how file size is measured.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Bitrate accounting in Figure 6 is underspecified: if I-frame costs, codebooks, and P-frame deltas are excluded from bpp, the parity claim against AV1/HEVC collapses.","rationale":"The reader's strongest_claim and weakest_assumption identify exactly the same load-bearing issue: the bpp accounting in Figure 6 is not derivable from the text. I read Section 3.8, Section 4.1, and Section 4.2 in the manuscript. Section 3.8 specifies quantizers symbolically (Qµ, Qℓ, Qc) with free parameters b, M, B and gives no entropy-coding step; Section 4.1 gives only splat counts and training details, not bit accounting. Section 4.2's strongest sentence, 'GSVC achieves better or comparable performance ... except VVC', depends entirely on Figure 6. The paper's own Section 5 caveat ('preliminary support of evidence') corroborates that the evaluation is not yet airtight. I find no internal inconsistency in the math itself — Eq. (3)–(5), (14)–(16) are coherent — and the reported 1583.72 fps decode speed is plausible for a Gaussian rasterizer, though it is a single number without a measurement protocol. The 90%-of-N bookkeeping is also slightly loose: if every P-frame inherits 90% of N and GSA adds 10%, the text says pruning brings the total back to 90% of N, which is consistent with a steady state of 90% of N per frame, but the I-frame uses 100% of N before pruning, so per-frame splat counts differ; this is minor compared to the rate-accounting gap. I also note the ablation figure (Figure 7) uses a different x-axis (Gaussian Number) than Figure 6 (bpp), so it cannot serve as a cross-check of the rate axis either. For the verdict, CONDITIONAL remains right: the paper's contribution — 2D Gaussian splats as a practical video representation — is plausible and novel enough to merit publication once the bitrate accounting is documented, but the headline comparative claim should not be accepted as-is without the missing parameters and a reproducible bit-counting procedure. The strongest claim also asserts a two-orders-of-magnitude decode speed advantage; that part is less fragile because it is a measured rendering throughput, but it is reported without error bars or hardware details beyond V100, so it is secondary to the rate-axis issue.","tokens_in":10652,"tokens_out":2195,"duration_ms":21094,"concrete_test":"Recompute the bpp for the reported operating points from Section 3.8 alone: use N = 10,000 splats/frame with the stated pruning (90% of N), an assumed GOP of 600 frames for UVG, and sum (a) I-frame 16-bit positions plus quantized ℓ and c′ codes, (b) P-frame coded deltas with the same quantizers, (c) RVQ codebook storage (M·B·3 values) amortized over the video, and (d) any entropy-coded residual if claimed. Vary b in {4, 8}, M in {1, 4}, B in {256, 1024}. If the lowest reconstruction of bpp exceeds the plotted 0.2103 bpp by more than 25%, the parity claim in Figure 6 is not supported; if it matches only under favorable unstated parameters, the paper should state those parameters and rerun Figure 6.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim of Section 4.2 — parity with AV1/HEVC and better-than-DCVC speed — rests on Figure 6's bitrate axis. Section 3.8 gives quantization operators but never states the quantization bit-widths b, the number of RVQ stages M, or the codebook size B, and never specifies whether the reported bpp includes: (i) the 16-bit float positions of the I-frame; (ii) the full I-frame cost amortized over the GOP length; (iii) the M codebooks themselves, which contain B×3 floats and are not frame-local; (iv) P-frame deltas of all three attributes (µ, ℓ, c′) including which components are differentially coded and with what precision; (v) any final entropy coding or arithmetic coder. The implementation-details paragraph only explains splat counts (10K–50K, pruning to 90% of N), so a rough cost check is impossible from the text. If, for example, b=8, M=2, B=256 and no entropy coding, the reported 0.2103–1.0715 bpp could undercount by a large factor, especially at low N where codebook overhead dominates. The manuscript itself calls the results 'preliminary support of evidence' (Section 5), acknowledging that the comparative claim is not fully established. Because Figure 6 is the only quantitative evidence for the headline parity claim, an underspecified rate axis is the single most load-bearing concern.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces GSVC, a video representation and compression framework based on 2D Gaussian splats. Each frame is represented by a set of 2D Gaussians with position, color, and Cholesky parameters. An I/P-frame structure is used: key frames are trained from scratch, while P-frames are initialized from the previous frame and fine-tuned. The method combines Gaussian Splat Pruning (GSP), Gaussian Splat Augmentation (GSA), and a Dynamic Key-frame Selector (DKS) to control rate, handle motion, and detect scene changes. Encoding quantizes parameters using 16-bit float positions, b-bit asymmetric quantization for Cholesky coefficients, and M-stage RVQ codebooks of size B for colors. Experiments on three UVG sequences report rate-distortion curves against AVC, HEVC, VVC, AV1, DCVC-DC, and DCVC-HEM, plus decoding speed of 1583.72 fps. The paper claims comparable rate-distortion performance to standard codecs in MS-SSIM and VMAF, except for VVC, while offering very fast decoding.","tokens_in":10980,"tokens_out":2472,"duration_ms":28038,"significance":"If the rate-distortion comparison is correct, GSVC is a promising explicit, editable representation for video with a useful speed advantage over neural codecs. The method's parameter controls (number of splats, pruning, convergence) are a practical strength, and the I/P-frame formulation is a natural extension of 2D Gaussian splatting. The paper also honestly discusses limitations and positions its results as preliminary. However, the central comparison rests on a rate axis whose bit accounting is not fully specified, and the evaluation is limited to three sequences without variance estimates. These issues currently prevent the parity claim from being verified, so the significance is conditional on a corrected and fully specified bitrate computation.","major_comments":[{"comment":"The bitrate computation underlying Figure 6 is underspecified. The paper defines quantization operators but never states the bit-depth b for Cholesky coefficients, the number of RVQ stages M, the codebook size B, or whether a final entropy coder is applied. It also does not explain how the reported bpp counts the 16-bit float positions of I-frames, the I-frame cost relative to GOP length, the learned codebooks (which are shared and not frame-local), the P-frame deltas for all three attributes, or any side information such as pruning masks and key-frame decisions. Because the claim of parity with AV1 and HEVC depends directly on the bpp values in Figure 6, the rate axis must be specified exactly enough for a reader to reproduce the cost. Please provide the missing parameters and a precise formula for bpp, including how codebooks and I-frame overhead are amortized.","section":"§3.8, Eq. (14)–(16) and Figure 6"},{"comment":"The evaluation uses only three sequences (Beauty, HoneyBee, Jockey) and reports single-run results without error bars or multiple trials. The training procedure is stochastic (random initialization, random GSA insertion, learned weights), so the reported curves may not be stable. Please report at least three independent runs per configuration with mean and variance, or justify why single runs are sufficient for this type of comparison.","section":"§4.1, Datasets and Implementation Details"},{"comment":"The central claim that GSVC achieves 'better or comparable performance in terms of MS-SSIM and VMAF... except VVC' is stated without quantitative backing. The text also admits lower PSNR in certain scenarios. Please specify at which bitrates and for which sequences the method is better, comparable, or worse, and report the actual metric differences (e.g., BD-rate or BD-PSNR relative to each codec). Without these numbers, the qualitative statement cannot be assessed, especially given the underspecified rate axis.","section":"§4.2, Rate-Distortion Trade-off"},{"comment":"The paper itself describes the results as 'preliminary support of evidence.' This is appropriate, but it should be reconciled with the stronger claims in the Abstract and Section 4.2. Please either temper the abstract and comparison statements to match the preliminary nature of the evidence, or provide the additional experiments and rate accounting needed to support the stronger claims.","section":"§5, Discussion and Conclusion"}],"minor_comments":[{"comment":"The paragraph beginning 'Recent efforts have explored Gaussian-based representations...' appears twice verbatim; please remove the duplicate.","section":"§2, Related Work"},{"comment":"The abbreviation for the GaussianImage baseline is written 'Outs-GI' in the baseline paragraph and 'Ours-GI' in the caption of Figure 6 and elsewhere; please use one consistent spelling.","section":"§4.1, Baseline"},{"comment":"The caption uses 'DSK' while the method is called 'DKS' in the text; clarify the intended acronym.","section":"Figure 5 caption"},{"comment":"The text mentions 'GuassianVideos' in 'the neural-based approach achieves significant improvements over GuassianVideos'; this appears to be a typo and should be corrected (e.g., to 'GaussianImage' or 'the Gaussian-based baselines' as appropriate).","section":"§4.2, Computational Cost"},{"comment":"The term 'bpp' is used without a formal definition in the context of GSVC; please define bits per pixel explicitly, including how the total bit count is computed from the encoded representation.","section":"§3.8 and §4.1"},{"comment":"The ablation text states that 'DSK has small improvements' whereas Figure 7 appears to show the curves for variants (1), (2), (3), and Ours; please clarify which variant includes DKS and report the corresponding numerical differences, since the figure alone does not clearly separate the DKS contribution.","section":"§4.3, Ablation Study"}],"recommendation":"major_revision","confidential_remarks":"The paper presents a novel and potentially useful representation, but the rate-distortion comparison is not yet verifiable because the bitrate accounting is incomplete. The evaluation scope is narrow, and the authors themselves label the results preliminary. I believe the central claims can be made solid with a full description of the rate computation, additional experimental detail, and more careful wording, so I recommend major revision rather than rejection. I would also encourage the journal to ask for code or a detailed breakdown of the encoded bitstream to enable reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: GSVC is a genuine and reasonably well-engineered first step toward using 2D Gaussian splats for video compression, but the headline parity claim against AV1/HEVC rests on a bitrate axis that the paper never fully specifies. I'd send it to review, but the authors need to nail down the rate accounting before the comparison can be believed.\n\nWhat's actually new: VGR and VeGaS used 3DGS for video but not with compression as a goal; GaussianImage and Image-GS did 2DGS for single images. GSVC is the first to combine I/P-frame prediction, contribution-based pruning, splat augmentation, and loss-based key-frame selection in a single 2DGS video codec. The ablation is useful: GSP does most of the work, GSA helps with dynamics, and DKS helps only at scene cuts (the UVG clips have none, so its small effect is expected). The explicit splat representation is a real plus: no pretraining, editable, and fast rendering. The 1500+ fps decode speed is believable for splat rendering and is two orders of magnitude above DCVC-DC.\n\nThe soft spot, as you identified, is Section 3.8/Figure 6. The paper states b-bit quantization for Cholesky and M-stage RVQ with codebook size B, then never gives b, M, or B, and never says whether the reported bpp includes the 16-bit float positions of I-frames, the amortized I-frame cost over the GOP, the codebooks (which are not frame-local), the P-frame deltas for all three attributes, or any final entropy coder. The implementation details only give splat counts (10K–50K, pruning to 90% of N). Without that, a reader cannot reconstruct the rate. If codebook overhead and I-frame costs are missing, the parity claim overstates. The authors call the results \"preliminary support of evidence\" in Section 5, which is honest but also an admission that the comparative claim is not fully established. Also, three sequences, no error bars, no code, and no specification of which UVG frames were used—all minor-to-moderate for a workshop-level paper but real for a claim about matching AV1/HEVC. The 90%-of-N vs. N inconsistency in Section 4.1 is a small but fixable confusion.\n\nWho this is for: researchers in Gaussian-based compression and learned video codecs. It is a solid system paper with a clear gap, not a breakthrough. I'd send it to a serious referee, specifically asking them to verify the rate calculation. If that checks out, the paper is acceptable; if not, it needs major revision or a rewrite of the claims.","headline":"GSVC is a real first step for 2D Gaussian video compression, but the rate accounting in Figure 6 must be pinned down before the AV1/HEVC parity claim can be trusted.","tokens_in":11521,"tokens_out":2586,"would_cite":false,"duration_ms":26034,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper tries to establish that 2D Gaussian splats can serve as a practical primitive for video compression, with rate-distortion performance comparable to AV1 and HEVC and decoding at over 1500 fps.","keywords":["2D Gaussian splatting","video compression","video representation","rate-distortion optimization","temporal prediction","key-frame selection","Gaussian splat pruning","Gaussian splat augmentation"],"falsifier":"Re-encode Beauty, HoneyBee, and Jockey with GSVC; serialize the quantized splat parameters, codebooks, I-frame data, and P-frame deltas into an actual file, apply any entropy coder the implementation uses, and plot the resulting bits-per-pixel against the published curves. If the true file size puts GSVC's PSNR/MS-SSIM/VMAF points to the right of the AV1 or HEVC curves at equal quality, the claimed rate-distortion parity fails.","tokens_in":10397,"feed_emoji":"🎬","tokens_out":9956,"duration_ms":90319,"temperature":0.7,"pith_summary":"This paper tries to establish that 2D Gaussian splats, already successful as an explicit primitive for static scenes and images, can also serve as an efficient primitive for video representation and compression. The proposed codec, GSVC, treats selected frames as key-frames learned from scratch and predicts the rest from the previous frame, then prunes low-contribution splats, injects new splats to absorb motion, and inserts a key-frame when a scene change is detected. On UVG clips, the paper reports rate-distortion curves comparable to AV1 and HEVC in perceptual metrics and decoding at roughly 1583 frames per second, orders of magnitude faster than neural video codecs. If the bitrate accounting is complete, this would give a practical, pretraining-free alternative to both block-based standards and neural codecs, with explicit control over rate, quality, and encoding time.","feed_headline":"Video as 2D Gaussian splats decodes at 1500 fps near AV1 quality","feed_subtitle":"GSVC claims AV1/HEVC-level rate-distortion from 2D Gaussian splats, with decoding above 1500 fps and no pretraining.","key_machinery":"The carrying object is the 2D Gaussian splat, an explicit primitive with eight parameters: position $\\mu\\in\\mathbb{R}^2$, weighted color $c'\\in\\mathbb{R}^3$ (opacity multiplied into the color), and Cholesky vector $\\ell\\in\\mathbb{R}^3$ defining the covariance $\\Sigma=LL^T$. Rendering accumulates $w_n c'_n \\exp(-\\sigma_n)$ over splats covering a pixel, where $\\sigma_n=\\frac{1}{2}d_n^T\\Sigma^{-1}d_n$ and $w_n$ is a learned importance weight. The mechanism has four moving parts: frame prediction reuses the previous frame's splats as initialization and stores only deltas; Gaussian Splat Pruning removes splats with small $|w_n|^2$; Gaussian Splat Augmentation inserts new splats with $w\\approx 0$ that can be activated by sharp weight changes; and the Dynamic Key-frame Selector flags a frame as a key-frame when its P-frame pretraining loss exceeds the I-frame loss by the local mean plus three local standard deviations. Encoding quantizes positions to 16-bit floats, Cholesky coefficients through $b$-bit asymmetric quantization, and colors through $M$-stage residual vector quantization.","core_discovery":"The central claim is that a video can be represented by a set of 2D Gaussian splats per frame, and that with temporal prediction this representation compresses competitively. Each splat carries eight parameters: a 2D position, a weighted RGB color with opacity folded in, and a three-coefficient Cholesky factor for its covariance, rendered by accumulated summation of weighted Gaussian contributions. GSVC encodes an I-frame from scratch and a P-frame as quantized deltas relative to the previous frame's splats, so temporal redundancy is stored once. A learnable importance weight per splat drives pruning; randomly inserted splats with near-zero initial weights capture newly appearing objects or fast motion; and frames whose P-frame loss exceeds the local I-frame loss by more than three local standard deviations become key-frames. The paper reports that this scheme reaches AV1/HEVC-comparable MS-SSIM and VMAF, with VVC remaining stronger, and decodes 1920x1080 content at 1583.72 fps, while neural codecs in the same comparison decode at around 2.4 fps.","pith_inferences":["The delta encoding used for P-frames is symmetric enough that the same machinery could support B-frames predicted from both past and future frames, which the paper names as future work; this should improve rate-distortion at the cost of latency.","Because splats are explicit objects, a region-of-interest mode is possible: allocate more splats where saliency or viewer attention is high, a knob block-based codecs do not expose.","The pruning mechanism implies a variable-rate mode: let N_t vary per frame under a quality threshold instead of holding N constant, which would mimic constant-quality coding and likely save bits on static stretches."],"forward_implications":["Because splats are explicit and countable, the number of splats N becomes a direct bitrate knob, the loss convergence threshold a quality knob, and the training iteration count an encoding-time knob.","Decoding at 1583 fps makes real-time playback and interactive editing feasible with an ordinary GPU rasterizer, without a specialized neural decoder.","GSVC requires no pretraining on a dataset, so it can in principle generalize to any input video, unlike the neural baselines compared in the paper.","The ablation ordering (GSP contributes most, then GSA, then DKS) indicates that where splats are placed matters more than the raw splat count.","Fine-grain progressive coding follows directly: pruned splats can be added back to refine quality without retraining from scratch."],"supporting_citations":[{"why":"Supplies the 2D Gaussian splat image primitive, the accumulated-blending renderer, and the attribute quantization-aware fine-tuning that GSVC builds on.","marker":"[Zhang et al.(2025)]"},{"why":"Establishes Gaussian splatting as a fast explicit scene representation and motivates the claimed decoding-speed advantage.","marker":"[Kerbl et al.(2023)]"},{"why":"Prior work applying 3D Gaussian splats to video; GSVC positions itself against this baseline.","marker":"[Sun et al.(2024)]"},{"why":"Concurrent video Gaussian splatting work that treats time as a third dimension, contrasted with GSVC's 2D frame-by-frame approach.","marker":"[Smolak-Dy˙zewska et al.(2024)]"},{"why":"H.264/AVC standard used as a rate-distortion baseline.","marker":"[Wiegand et al.(2003)]"},{"why":"HEVC standard used as a baseline; the paper claims comparable MS-SSIM/VMAF against it.","marker":"[Sullivan et al.(2012)]"},{"why":"VVC standard used as the strongest codec baseline, which GSVC does not claim to match.","marker":"[Sullivan(2020)]"},{"why":"AV1 codec overview; AV1 is the main non-VVC standard GSVC claims to match.","marker":"[Chen et al.(2020)]"},{"why":"DCVC neural video codec used as the neural baseline; its 2.4 fps decode speed contrasts with GSVC's 1583 fps.","marker":"[Li et al.(2023)]"},{"why":"Provides the UVG dataset clips (Beauty, HoneyBee, Jockey) on which all rate-distortion curves are measured.","marker":"[Mercat et al.(2020)]"}],"fun_headline_variants":["Video compressed as 2D Gaussian splats hits 1500 fps decode","Gaussian splats compress video like AV1, decode at 1500 fps","2D splat video codec rivals AV1, runs 1500 fps","Video via 2D Gaussian splats: AV1-grade quality at 1500 fps","Splat-based video codec targets AV1, decodes at 1500 fps"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the bits-per-pixel values plotted for GSVC in Figure 6 include every bit needed to reconstruct the video, including the quantized splat positions, Cholesky coefficients, color codebooks and their indices, I-frame data, and P-frame deltas, because the paper never states the quantization bit-width $b$, the number of RVQ stages $M$, the codebook size $B$, or whether a final entropy coder is applied.","fun_headline_variants_meta":{"raw":{"variants":["Video compressed as 2D Gaussian splats hits 1500 fps decode","Gaussian splats compress video like AV1, decode at 1500 fps","2D splat video codec rivals AV1, runs 1500 fps","Video via 2D Gaussian splats: AV1-grade quality at 1500 fps","Splat-based video codec targets AV1, decodes at 1500 fps"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000626,"raw_usage":{"total_tokens":2925,"prompt_tokens":1001,"completion_tokens":1924,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":617,"completion_tokens_details":{"reasoning_tokens":1816}},"tokens_in":617,"tokens_out":1924,"duration_ms":13018,"temperature":1.0,"reasoning_tokens":1816,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T17:33:13.053686+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-encode Beauty, HoneyBee, and Jockey with GSVC; serialize the quantized splat parameters, codebooks, I-frame data, and P-frame deltas into an actual file, apply any entropy coder the implementation uses, and plot the resulting bits-per-pixel against the published curves. If the true file size puts GSVC's PSNR/MS-SSIM/VMAF points to the right of the AV1 or HEVC curves at equal quality, the claimed rate-distortion parity fails.","supporting_citations":[],"review_version":1}