{"id":"4328d4c2-163b-4083-be4a-e5de0914ba7d","arxiv_id":"2501.00975","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"CoordFlow represents videos as a soft-weighted blend of layers, each with its own similarity-motion compensation network, achieving state-of-the-art PSNR per bit among pixel-wise INR codecs.","lead":"A new pixel-wise neural video representation splits each frame into motion-compensated layers, each handled by a small network, and then blends them with a learned attention map. It reports the best rate-distortion trade-off among pixel-wise implicit neural representations and near-parity with the leading frame-wise method on common benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"BPP comparison in Table 3 is not apples-to-apples: CoordFlow's BPP uses 8-bit color-only quantization while NVP's BPP reflects a different pipeline; the claimed 'superior to NVP in both BPP and PSNR' needs the same compression protocol to hold.","rationale":"The reader identified the rigidity of the similarity-transform flow model (Eq. 1, Appendix B) as the load-bearing assumption. That is a legitimate modeling limitation, but it is not the most load-bearing point for the paper's central claim, because the paper's headline is an empirical SOTA claim, not a claim that the model generalizes to all real-world motions. The ablation study (Table 5) directly addresses the modeling premise: removing the flow network costs 1.72 dB on average, and the two-layer structure is explicitly recommended, so the premise is tested internally. The weaker spot is the BPP comparison. The paper's own tables contain an inconsistency that directly affects the 'superior to NVP' and 'first pixel-wise method to outperform frame-wise' claims: Table 3 lists CoordFlow S at 0.08 BPP with no quantization protocol, while Table 4 shows 0.02 BPP after a partial (color-only, 8-bit) quantization. The baseline NVP BPP of 0.90 is taken from another paper with its own pipeline. If the comparison is not protocol-matched, the rate-distortion advantage may be an artifact. This is a concrete, checkable correctness risk, more decisive than the modeling assumption, since a fair BPP comparison would settle the central claim. I therefore recommend keeping the CONDITIONAL verdict, but for a different reason than the reader's; the condition should be a protocol-matched BPP/PSNR comparison with code release, not merely a multiple-seed rerun.","tokens_in":11882,"tokens_out":1753,"duration_ms":14095,"concrete_test":"Reproduce Table 3 using an identical compression protocol for CoordFlow and NVP: quantize the entire model (including flow network and any auxiliary weights) to the same bit width with the same entropy coder, and report average PSNR over UVG at matched BPP points (e.g., 0.02, 0.05, 0.10, 0.20, 0.50). If CoordFlow remains above NVP's rate-distortion curve at every matched BPP, the pixel-wise SOTA claim stands; if the 0.08 BPP entry comes from a different protocol, the claim needs revision.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central performance claim is that CoordFlow is 'the first pixel-wise method to outperform frame-wise INRs' and sets a new pixel-wise SOTA (Section 3.1, conclusion). Table 3 is the only direct pixel-wise BPP/PSNR comparison, and it is internally inconsistent: CoordFlow S is listed at 0.08 BPP with PSNR 34.23, while Table 4 reports CoordFlow S at 0.02 BPP after 8-bit quantization of only the color network (98.3% of weights). The Table 3 numbers therefore appear to use a different, unspecified bit allocation (the main text says 'no quantization' but Table 3 has BPP values, so some quantization/entropy coding must have occurred). NVP's 0.90 BPP and 35.79 PSNR are taken from the original NVP paper, which uses a specific quantization and entropy-coding scheme. Comparing a partially quantized CoordFlow against NVP's full pipeline biases the BPP comparison. Figure 3 also mixes results from [17, 15, 13, 18] without stating the quantization protocol per curve, so the visual claim of superiority over NVP is not verifiable. The central 'first pixel-wise method to beat frame-wise INR' claim depends on this comparison being fair; if CoordFlow's BPP were computed with the same full-model quantization and entropy coding as the baselines, its PSNR-BPP advantage might shrink or reverse.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CoordFlow, a pixel-wise implicit neural representation (INR) for video compression. The architecture is composed of multiple CoordFlow layers, each pairing a flow network that predicts a per-frame similarity transform (scale, rotation, translation) with a color network that maps transformed (x,y,t) coordinates to RGBA values; the layers are combined through softmax-weighted alpha blending. The method is evaluated on the UVG and Boat datasets, with PSNR comparisons against frame-wise and pixel-wise INRs, a BPP/PSNR comparison, an ablation study, and qualitative demonstrations of video upsampling, unsupervised segmentation, inpainting, stabilization, and denoising. The paper claims state-of-the-art performance among pixel-wise INRs, superiority over NVP in both BPP and PSNR, and that CoordFlow is the first pixel-wise method to outperform frame-wise INRs.","tokens_in":12211,"tokens_out":8679,"duration_ms":77976,"significance":"If the empirical claims are confirmed, CoordFlow would be a meaningful advance: it offers a simple and interpretable way to inject global motion compensation into pixel-wise INRs, produces unsupervised layer segmentation as a byproduct, and reaches PSNR levels close to strong frame-wise methods. The ablation study in Section 3.3 is encouraging because it isolates the contribution of the multi-layer structure and the flow network. However, the rate-distortion comparison contains a serious protocol inconsistency, the training budgets are unequal, and the headline claim is broader than the data support. The significance is therefore conditional on a re-run of the comparison under a common and fully documented compression pipeline.","major_comments":[{"comment":"The BPP/PSNR comparison is not rate-distortion controlled and is internally inconsistent. Table 3 is described as reporting results with \"no quantization\", yet it lists BPP values; CoordFlow S appears there with PSNR 34.23, whereas Table 1 reports 34.40 for the same raw model. Table 4 then reports lower BPP values after quantizing only the color network (about 98.3% of weights) to 8-bit, leaving the flow network and any side information out of the reported bit budget. The baselines' BPP values are taken from [13, 18], which use their own full quantization and entropy-coding pipelines. The claimed superiority over NVP in both BPP and PSNR is therefore not established. Please report CoordFlow's BPP under the same full-model quantization and entropy-coding protocol as the baselines, including all weights and side information, or explicitly state that the comparison is not a controlled rate-distortion comparison.","section":"Section 3.1, Tables 3 and 4; Section 3.2"},{"comment":"The headline PSNR comparison in Table 1 is confounded by unequal training budgets. The text states that all CoordFlow models were trained for 53 epochs, while the table caption says frame-wise models were tested after 300 training epochs and the baseline numbers are imported from other papers. Since INR quality is strongly dependent on training length and optimization details, a comparison at 53 versus 300 epochs does not establish that CoordFlow is better at equal training effort. In addition, single runs with no error bars or multiple seeds make it impossible to assess the stability of the reported PSNR values. Please provide convergence curves, or retrain all compared methods under the same epoch and seed protocol.","section":"Section 3, training details, and Table 1"},{"comment":"The claim that CoordFlow is \"the first pixel-wise method to outperform frame-wise INRs\" is not supported by the paper's own tables. Table 1 and Appendix Table 8 show HiNeRV, a frame-wise method, outperforming CoordFlow at every size on the UVG average (e.g., S: 35.27 vs 34.40; M: 36.71 vs 35.33; L: 38.02 vs 36.54). The supported statement is that CoordFlow outperforms the NeRV-family baselines considered here, not all frame-wise INRs. Please qualify the claim to match the evidence, for example by saying it outperforms the tested NeRV-family baselines and is competitive with the leading frame-wise method HiNeRV.","section":"Section 5, Conclusion; Table 1; Appendix Table 8"}],"minor_comments":[{"comment":"The manuscript contains numerous grammatical and typographical errors, including \"it's\" for \"its\", \"out preforming\" for \"outperforming\", \"secures it's competitiveness\" for \"secures its competitiveness\", and \"tree different model sizes\" for \"three different model sizes\"; a careful proofreading pass is needed.","section":"General"},{"comment":"The combined loss is written as w * (||δ||₁ + λ||δ||₂), which mixes an L1 norm with an unsquared L2 norm; the text refers to the second term as an L2 loss. Please clarify whether the L2 term is squared, since this changes the effective balance between the two terms and affects reproducibility.","section":"Section 2.3, Equation (5)"},{"comment":"The per-pixel loss weight w is described only qualitatively in terms of Laplacian magnitude, Canny edge prominence, and temporal variance, but no explicit formula is given. Please provide the exact definition so that the training loss is reproducible.","section":"Section 2.3"},{"comment":"The flow model assumes a per-layer global similarity transform, and Appendix B explicitly states that a single layer is \"not designed for capturing intricate real-world motions\". The paper should state more clearly how the two-layer version handles non-rigid motion, occlusions, or perspective changes, and discuss the extent to which the color network must memorize residuals in such cases, since this affects the compression-efficiency claim.","section":"Section 4.2 and Appendix B"},{"comment":"Figure 3 mixes results from several sources ([17, 15, 13, 18]) without stating which quantization or entropy-coding protocol each curve uses, and the axes are not self-explanatory in the current caption. Please specify the protocol for each curve and label the axes clearly.","section":"Figure 3"},{"comment":"Table 6 reports training duration per epoch, but the actual total training time depends on the number of epochs (53 in the main experiments) and on the video length; please report total encoding time so the reader can assess the practical cost.","section":"Appendix D"}],"recommendation":"major_revision","confidential_remarks":"To the editor: the architectural idea is interesting and the ablation is encouraging, but the central rate-distortion comparison is the main weakness. I recommend requiring a re-run of the BPP/PSNR evaluation under a consistent full-model quantization and entropy-coding protocol, and a careful rewording of the headline claim, before considering acceptance. The paper would also benefit from releasing code or a detailed training configuration to make the results reproducible."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: CoordFlow is a sensible pixel-wise INR that uses layered similarity transforms to exploit temporal redundancy, and its headline PSNR numbers actually look strong. But the paper overstates its place in the literature, and the BPP comparison in Table 3 is muddled enough that the compression claims need a redo.\n\nThe genuinely new thing is applying the Layered Neural Atlases idea to pixel-wise INR compression, with per-frame similarity transforms rather than homographies. That is a modest novelty, but the empirical result is meaningful: at 12.68M parameters CoordFlow L hits 36.54 dB average on UVG, higher than the 136M-parameter NVP at 35.83 dB and above NeRV, E-NeRV, HNeRV, and FFNeRV. The ablation is clean—removing the flow network and extra layer costs about 1.7 dB—so the components are doing real work. The byproducts (unsupervised segmentation, inpainting, stabilization, upsampling) are plausible and well-illustrated. Training for 53 epochs versus 300 for the frame-wise baselines is also a plus.\n\nThe soft spots are in the claims and the evaluation protocol. The conclusion says this is 'the first pixel-wise method to outperform frame-wise INRs,' but Table 1 and Appendix E show HiNeRV still beats CoordFlow by 0.9–1.5 dB depending on size. 'On-par' in the abstract is generous; 'approaches HiNeRV' would be accurate. More importantly, Table 3 is titled 'no quantization' but reports BPP values that do not correspond to a simple raw-parameter count, and the NVP BPP is taken from a different paper with its own quantization and entropy-coding pipeline. Comparing those numbers as if they were the same protocol is not sound. This needs to be recomputed with the same full-model quantization and entropy coding for all methods. Add to that the lack of error bars, single runs, baselines pulled from prior papers, and no code release—the performance claims are plausible but not yet reproducible. The modeling limitation of a few similarity-moving layers is acknowledged in Appendix B and is not a fatal flaw; the method just won't handle arbitrarily complex motion.\n\nFor a serious referee: yes, this deserves peer review. The architecture is simple, the ablation is informative, and the raw PSNR results are competitive. But I'd send it back for major revision: fair BPP comparison, multiple seeds, code or at least detailed hyperparameters, and a rewritten conclusion that doesn't claim more than the tables show. I would not cite it in its current form, but I'd watch for the revised version.","headline":"CoordFlow has a solid pixel-wise INR design and strong PSNR numbers, but its compression claims rest on an apples-to-oranges BPP comparison and an overbroad 'first to beat frame-wise' statement.","tokens_in":12721,"tokens_out":5284,"would_cite":false,"duration_ms":46496,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CoordFlow claims to be the first pixel-wise neural video representation to outperform frame-wise INRs, by splitting the video into motion-compensated layers with a similarity transform per layer before color reconstruction.","keywords":["CoordFlow","implicit neural representation","video compression","pixel-wise INR","motion compensation","similarity transformation","unsupervised segmentation","rate-distortion"],"falsifier":"Train CoordFlow on a synthetic scene with two independent non-rigid motions, such as a waving flag in front of a slowly panning background, at a fixed parameter count, and compare PSNR against the same model trained on a rigid two-motion scene. If the non-rigid version's advantage over a single-layer color-only baseline shrinks to near zero, the similarity-flow layer is not carrying the compression gain; if the per-layer flow matrices fail to align the moving regions, the layered-motion premise fails.","tokens_in":11675,"feed_emoji":"🎬","tokens_out":8412,"duration_ms":69645,"temperature":0.7,"pith_summary":"CoordFlow is a neural video-compression method that reconstructs each pixel from its $(x,y,t)$ coordinates rather than generating whole frames from a time index. It splits the video into a small number of layers, typically a background and a foreground, and gives each layer a flow network that applies a single similarity transform (scale, rotation, translation) per frame plus a color network that paints the stabilized content. The paper's central claim is that this decomposition lets pixel-wise representations close the gap with frame-wise ones: on the UVG-HD benchmark, the largest CoordFlow model averages 36.54 dB PSNR with 12.68M parameters, ahead of every pixel-wise baseline and every frame-wise model except HiNeRV. Because the flow is explicit, the same representation yields unsupervised segmentation, upsampling and frame interpolation, inpainting, stabilization, and denoising as byproducts. If the claim holds, CoordFlow reopens the pixel-wise route to video INRs, a route that had lagged behind frame-wise designs on rate-distortion.","feed_headline":"Pixel-wise video network now beats frame-wise rivals","feed_subtitle":"CoordFlow splits video into moving layers, beating every pixel-wise baseline and all frame-wise methods but one on UVG.","key_machinery":"The central object is the CoordFlow layer: a flow network that reads only the time coordinate $t$ and outputs a similarity-transformation matrix, and a color network that evaluates RGB$\\alpha$ at the transformed coordinates. The matrix is restricted to scale, rotation, and translation so that each layer represents one coherent motion per frame; with two or more layers in parallel, a softmax over $\\alpha$ values lets the model split the video into regions whose motions are separately aligned. The mechanism's work is to create a canonical space in which the color network sees nearly static content, so its capacity goes into appearance instead of memorizing frame-to-frame displacement. A layer-specific loss, multiplied by the layer's softmax $\\alpha$, drives the segmentation to converge so that the layer best able to stabilize a region takes responsibility for it.","core_discovery":"The paper proposes that a video's temporal redundancy can be removed by a per-layer similarity transform before a color network sees any coordinates. Each CoordFlow layer computes a $2\\times 3$ matrix from the time coordinate $t$, with entries built from scale $s$, rotation $\\theta$, and translation $(\\Delta x,\\Delta y)$; this matrix realigns $(x,y)$ into a canonical space, and a separate color network outputs RGB and $\\alpha$ for the stabilized coordinates while still receiving $t$ to absorb residual appearance changes. A softmax over the layers' $\\alpha$ values blends their outputs, so training discovers an unsupervised foreground/background segmentation, and a per-layer loss weighted by $\\alpha$ pushes each layer to specialize on the motion it can actually stabilize. The authors report that with two layers, CoordFlow's small, medium, and large versions all outperform every pixel-wise INR they compared with and all frame-wise INRs except HiNeRV in average PSNR on UVG-HD at comparable parameter counts; after 8-bit quantization of the color network, the large model reaches 36.19 dB at 0.07 bpp. The conclusion drawn is that CoordFlow is the first pixel-wise method to outperform frame-wise INRs.","pith_inferences":["[Editorial inference] If the layered-similarity hypothesis is the real source of gain, extending the flow network to output an affine or homography matrix per layer per frame should further improve rate-distortion on scenes with perspective change; this is a testable extension the paper does not run.","[Editorial inference] The alpha maps produced by a two-layer CoordFlow could serve as a self-supervised object-discovery signal for other video tasks, since the segmentation emerges without any flow supervision or mask labels.","[Editorial inference] The denoising and stabilization behaviors suggest a practical self-supervised cleanup recipe: train CoordFlow on a noisy or shaky video, then decode with smoothed flow matrices and high-frequency-free sampling rather than training a separate filter.","[Editorial inference] The 'one motion per frame per layer' restriction means performance on complex scenes should degrade as non-rigid content grows; a natural stress test is measuring how average PSNR per parameter drops as the number of independent moving objects grows beyond the layer count."],"forward_implications":["On UVG-HD, CoordFlow's large model scores 36.54 dB average PSNR at 12.68M parameters, above the 136M-parameter NVP baseline (35.83 dB) and above every frame-wise model tested except HiNeRV (38.02 dB).","Quantizing only the color network to 8-bit integers, CoordFlow L still delivers 36.19 dB at 0.07 bpp, a markedly lower bitrate than the uncompressed pixel-wise baselines reported for NVP.","On the Boat video, CoordFlow with 1.56M parameters reaches 32.67 dB PSNR, exceeding HiNeRV's 31.25 dB at 3.08M parameters.","The representation's explicit flow matrices enable video stabilization by temporal smoothing of the transformation matrices, unsupervised segmentation from the alpha maps, and inpainting by reading the background-only layer.","Because the input space is continuous, a single trained CoordFlow model can be sampled at new spatial positions and new times, giving upsampling and frame interpolation from one set of weights."],"supporting_citations":[{"why":"Introduces NeRV, the first frame-wise video INR and the primary frame-wise baseline CoordFlow compares against.","marker":"[7]"},{"why":"NVP is the pixel-wise state-of-the-art baseline CoordFlow aims to surpass and the source of several comparison results.","marker":"[13]"},{"why":"HiNeRV is the frame-wise benchmark that remains ahead of CoordFlow on average UVG PSNR and the source of most frame-wise comparison numbers.","marker":"[15]"},{"why":"FFNeRV is a flow-guided frame-wise INR whose motion-compensation design motivates CoordFlow's flow component.","marker":"[17]"},{"why":"SIREN is the classic pixel-wise INR baseline with sinusoidal activations, used as a comparison point and architectural ancestor.","marker":"[25]"},{"why":"Fourier feature networks provide the pixel-wise FFN baseline and supply the positional encoding scheme used in CoordFlow.","marker":"[28]"},{"why":"Layered neural atlases supply the layered-atlas concept of representing a video as separate networks for content and warping.","marker":"[12]"},{"why":"The UVG-HD dataset is the benchmark on which all CoordFlow rate-distortion results are measured.","marker":"[19]"}],"fun_headline_variants":["CoordFlow: first pixel-wise video INR to beat frame-wise","Pixel-wise video INR with moving layers rivals frame-wise","CoordFlow: pixel-wise video codec beats all frame-wise but one","CoordFlow splits video into moving layers for better compression"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes a video can be split into a few layers, each moving under a single global similarity transform per frame; non-rigid motion, occlusions, and perspective changes violate this, forcing the color network to memorize the residuals and raising the bitrate.","fun_headline_variants_meta":{"raw":{"variants":["CoordFlow: first pixel-wise video INR to beat frame-wise","Pixel-wise video INR with moving layers rivals frame-wise","CoordFlow: pixel-wise video codec beats all frame-wise but one","CoordFlow splits video into moving layers for better compression"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000885,"raw_usage":{"total_tokens":3836,"prompt_tokens":972,"completion_tokens":2864,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":588,"completion_tokens_details":{"reasoning_tokens":2804}},"tokens_in":588,"tokens_out":2864,"duration_ms":22155,"temperature":1.0,"reasoning_tokens":2804,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:37:38.264019+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train CoordFlow on a synthetic scene with two independent non-rigid motions, such as a waving flag in front of a slowly panning background, at a fixed parameter count, and compare PSNR against the same model trained on a rigid two-motion scene. If the non-rigid version's advantage over a single-layer color-only baseline shrinks to near zero, the similarity-flow layer is not carrying the compression gain; if the per-layer flow matrices fail to align the moving regions, the layered-motion premise fails.","supporting_citations":[{"cited_title":"Advances in Neural Information Processing Systems 34, 21557–21568 (2021)","cited_arxiv_id":null,"evidence_quote":"Introduces NeRV, the first frame-wise video INR and the primary frame-wise baseline CoordFlow compares against."},{"cited_title":"Advances in Neural Information Processing Systems 35, 12718–12731 (2022)","cited_arxiv_id":null,"evidence_quote":"NVP is the pixel-wise state-of-the-art baseline CoordFlow aims to surpass and the source of several comparison results."},{"cited_title":"Advances in Neural Information Processing Systems 36 (2024)","cited_arxiv_id":null,"evidence_quote":"HiNeRV is the frame-wise benchmark that remains ahead of CoordFlow on average UVG PSNR and the source of most frame-wise comparison numbers."},{"cited_title":"In: Proceedings of the 31st ACM International Conference on Multimedia","cited_arxiv_id":null,"evidence_quote":"FFNeRV is a flow-guided frame-wise INR whose motion-compensation design motivates CoordFlow's flow component."},{"cited_title":"Advances in neural information processing systems 33, 7462–7473 (2020)","cited_arxiv_id":null,"evidence_quote":"SIREN is the classic pixel-wise INR baseline with sinusoidal activations, used as a comparison point and architectural ancestor."},{"cited_title":"Advances in Neural Information Processing Systems 33, 7537–7547 (2020)","cited_arxiv_id":null,"evidence_quote":"Fourier feature networks provide the pixel-wise FFN baseline and supply the positional encoding scheme used in CoordFlow."},{"cited_title":"ACM Transactions on Graphics (TOG) 40(6), 1–12 (2021)","cited_arxiv_id":null,"evidence_quote":"Layered neural atlases supply the layered-atlas concept of representing a video as separate networks for content and warping."}],"review_version":1}