{"id":"1e70361c-2f09-4d81-964f-d42226bb4c5d","arxiv_id":"2504.12165","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"CodingHomo combines codec motion vectors with masked image features to estimate homographies, setting a new state of the art on the CA-unsup and GHOF benchmarks within its small-baseline scope.","lead":"CodingHomo reads the motion vectors already stored inside H.264 compressed videos and uses them as a free hint to align image pairs. In tests, it estimates the dominant-plane transformation more accurately than prior unsupervised methods, especially in scenes with moving objects and low texture.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"GHOF result may rely on leaking correspondence ground truth into mask supervision; cross-dataset generalization claim needs a train/test separation check.","rationale":"The reader correctly identified the large-baseline failure mode and the under-specification of GHOF MV generation as serious concerns, and I agree with the CONDITIONAL verdict. My proposal is sharper and more specific: the central SOTA claim on GHOF is a generalization claim, but the paper never explicitly states that GHOF was excluded from training or hyperparameter selection. The GHOF dataset is described as having a training set (approximately 10,000 frames) and an evaluation set (246 pairs), and the paper says only that MVs were added to the test set, not that the network was trained exclusively on CA-unsup. Moreover, alpha = 0.2 is selected on the CA-unsup test set, and the GHOF numbers are presented without a train/test separation statement, so the headline robustness claim is not yet falsifiable from the manuscript text alone. This is an internal consistency and reporting gap rather than a disagreement with consensus; the paper's own limitation section (Sec. V) admits that large-baseline MVs can be all-zero, yet the GHOF results show large gains precisely in fog/rain/snow. The proposed retraining check would settle whether the MV prior is genuinely useful across domains or whether the GHOF gains are an artifact of dataset overlap or tuning. I also acknowledge the paper's independent support: ablations, a reasonable baseline set, and an honest limitation statement all point to a basically sound method; my concern is about the strength of the cross-dataset claim, not about the method's existence or internal coherence.","tokens_in":21106,"tokens_out":1440,"duration_ms":13522,"concrete_test":"Retrain the full CodingHomo pipeline using only CA-unsup training pairs, with no GHOF frames anywhere in training, validation, or hyperparameter selection; then evaluate on the GHOF test set using the same MV extraction. If the avg PME remains near 1.21, the generalization claim holds. If it degrades toward or beyond the RealSH value of 1.72, the reported GHOF numbers depend on GHOF training data and must be relabeled as in-domain results.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The GHOF evaluation is presented as a generalization experiment, but the paper does not state whether the mask generators or loss were trained on GHOF. Section IV-A says GHOF 'includes a training set of approximately 10,000 video frames...' and Section IV-C2 says 'we added the corresponding MVs to the test set (GHOF)' but never says the model was trained only on CA-unsup. If GHOF training frames were used to tune the network, the NLL loss in Eq. 12 or the MGF mask generator could learn GHOF-specific MV/homography statistics, undermining the 'unseen fog, rain, and snow' claim. The paper's own largest gain on GHOF (29.65% average PME improvement, from 1.72 to 1.21) is exactly on those unseen-condition categories where MVs encode encoder block search artifacts rather than true motion; without an explicit train/test protocol statement, the cleanest explanation is dataset leakage or overfitting to GHOF statistics, not a principled motion prior.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents CodingHomo, an unsupervised deep homography estimation framework that augments image pairs with motion vectors extracted from H.264/AVC video bitstreams. The main technical contributions are a Mask-Guided Fusion (MGF) module that fuses a coarse homography with motion vectors under a learned motion-rejection mask, a Mask-Guided Homography Estimation (MGHE) module that uses the mask in coarse-to-fine refinement, and an Enhanced Motion Mask (EMM) that combines a coplanarity-aware mask with a motion-rejection mask for loss weighting, trained with a negative log-likelihood loss. Experiments on the CA-unsup dataset report state-of-the-art average point matching error (0.31 PME) and a generalization experiment on the GHOF dataset reports a larger improvement (1.21 PME), with qualitative results on fog, rain, and snow scenes. Code and dataset are promised publicly.","tokens_in":21327,"tokens_out":6371,"duration_ms":66529,"significance":"If the empirical results are reproducible, this is a novel and useful direction: it is the first unsupervised homography framework to exploit codec motion vectors as a prior, and the engineering is solid in several respects. The paper includes comparisons against feature-based, supervised, and unsupervised baselines on two datasets, per-component ablations, standard deviations, a t-test, robustness curves, and a public code/data commitment. These strengths make the central claim clearly testable. The main risks are protocol-level: the GHOF generalization experiment does not state whether GHOF training frames were used in any way, and the hyperparameter α is selected on the test set. Both are fixable with clarifications or additional experiments, so the paper is best treated as a major revision rather than a rejection.","major_comments":[{"comment":"The GHOF generalization experiment lacks an explicit training protocol. The paper reports GHOF as a generalization test and emphasizes improvements on unseen fog, rain, and snow scenes, but Section IV-A states that GHOF includes a training set of approximately 10,000 frames, and Section IV-C2 only says \"we added the corresponding MVs to the test set (GHOF)\". It is never stated whether the network, the mask generators, or the hyperparameter α were trained or tuned on any GHOF frames. Since the largest reported relative gain (1.72 to 1.21, 29.65%) occurs on the GHOF categories, this omission leaves open the possibility that the improvement reflects learning GHOF-specific MV/homography statistics rather than a generalizable motion prior. Please state definitively which GHOF data were used for training, validation, and model selection; if none were used, say so explicitly.","section":"Section IV-C2 / Section IV-A"},{"comment":"The floor parameter α in Eq. (6) is selected on the CA-unsup test set. Table IV reports PME on the test set for α ∈ {1.0, 0.4, 0.2, 0.0}, and the final model uses α = 0.2 from that table. This is test-set model selection, so the PME values in Table I and the t-test against RealSH (t = 3.7102) are obtained from a model whose hyperparameter was tuned on the same test set, while the comparison methods were not afforded such tuning. Please introduce a validation split, select α on it, and report the corresponding test numbers, or report all α variants on a held-out validation set to establish that the SOTA claim is not an artifact of test-set tuning.","section":"Section IV-F / Table IV"},{"comment":"The statement \"All learning-based methods are trained on the CA-sup dateset\" is inconsistent with the rest of the manuscript, where CAHomo, BasesHomo, and HomoGAN are described as unsupervised methods trained on CA-unsup. If some comparison methods were trained on a different dataset (for example RealSH on CA-sup), the comparison in Tables I and II is not controlled; if the sentence is a typo, it should be corrected. Please specify the exact training data and protocol for every learning-based baseline, including RealSH and the unsupervised methods.","section":"Section IV-C1"}],"minor_comments":[{"comment":"Please fix the typo \"A VC/H.264\" (should be \"AVC/H.264\") and the repeated \"video video coding\" in the same section.","section":"Section III-B"},{"comment":"The row labeled \"I3×3\" is not defined in the text or caption; please state explicitly that it is the identity-transform baseline.","section":"Table I"},{"comment":"The caption says \"Red block donates a dynamic vehicle\"; \"donates\" should be \"denotes\".","section":"Figure 2 caption"},{"comment":"The encoder configuration used to generate the H.264 bitstreams and MVs for GHOF is not reported. The paper only says MVs were \"added\" to GHOF; without encoder settings (QP, search range, frame structure, minimum block size, reference-frame count), the motion-vector input is not reproducible.","section":"Section IV-A / IV-C2"},{"comment":"The mask notation is inconsistent: \"Mi−1m\", \"Mi m\", and \"fMi−1m\" are used without a uniform convention. Please define the superscript/subscript placement clearly and use it consistently through Eqs. (5)–(7).","section":"Section III-D"}],"recommendation":"major_revision","confidential_remarks":"The GHOF protocol ambiguity is the most important issue. If the authors cannot confirm that no GHOF training frames or validation frames were used anywhere in training or model selection, the generalization claims in Section IV-C2 should be substantially reframed. The test-set tuning of α also undermines the reported t-test; this should be fixed with a validation split before the SOTA claim is accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: CodingHomo is the first unsupervised homography method to consume H.264 motion vectors, and the gains it reports look real. On CA-unsup the improvement over RealSH is modest (0.31 vs 0.34) but consistent across categories, and the GHOF generalization numbers are strong if the setup is what it appears to be. This paper deserves a serious referee, not a desk reject.\n\nWhat's new: the MGF and MGHE modules are a sensible way to fuse a block-level MV prior with learned features, and the NLL-based mask loss is a neat trick to keep the motion-rejection mask honest. The ablation table supports each component, and the authors state the main limitation plainly: if true motion exceeds the encoder's search range, the MV prior collapses and the method gets no lift. That candor is worth acknowledging.\n\nSoft spots, in decreasing order of importance. First, the GHOF experiment is underspecified. The paper says they \"added the corresponding MVs to the test set,\" but it never states in one place whether the model was trained exclusively on CA-unsup or whether GHOF training frames touched the network or mask generators. The GHOF training set is described in the dataset section, which makes the omission jarring. I don't think the leakage interpretation is the \"cleanest explanation\" — the natural reading is that only the CA-unsup model was tested — but the authors need to state the protocol explicitly. Second, alpha in Eq. 6 is selected on the CA-unsup test set via Table IV. That is arguably test-set tuning and should be moved to a validation split or defended as a robustness check. Third, MV generation for GHOF is under-described: no encoder settings, QP, or partition constraints, so the reader cannot reproduce the MV inputs. Fourth, minor: the GitHub link in the abstract is mangled and there is no commit hash.\n\nThe t-test and standard deviations are good practice. Self-citation is heavy but the cited methods are the actual baselines, so it is not a red flag. The stress-test worry about block-search artifacts in fog and rain is speculative; MVs in those conditions are noisy but still carry plane motion, and the results are not obviously explained by leakage.\n\nRecommendation: send to peer review. Ask for a clear training-protocol statement for GHOF, validation-based hyperparameter selection or a sensitivity analysis, and full MV-extraction details.","headline":"Solid niche advance using H.264 MVs for unsupervised homography; credible but needs a clearer GHOF training protocol and validation-based alpha.","tokens_in":21808,"tokens_out":3250,"would_cite":true,"duration_ms":32325,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Motion vectors already embedded in H.264 video, normally discarded at decode time, can be fused with image features through learned masks to give unsupervised homography its best reported accuracy: 0.31 on CA-unsup and 1.21 on GHOF.","keywords":["homography estimation","unsupervised learning","motion vectors","H.264/AVC","video coding prior","mask-guided fusion","dominant plane estimation","image alignment"],"falsifier":"Replace the H.264 motion-vector input with a scrambled or all-zero flow on the CA-unsup test set and compare average point-matching error: if it remains near 0.31 the motion prior is not load-bearing, and if it jumps toward the 0.36-0.46 range seen in the no-mask ablations, the prior is confirmed as the source of the gain.","tokens_in":20906,"feed_emoji":"🎥","tokens_out":7903,"duration_ms":70391,"temperature":0.7,"pith_summary":"Motion vectors computed by an H.264 encoder to compress video are usually thrown away after decoding; this paper argues that they are a free, reliable prior for estimating the homography between consecutive frames. CodingHomo is an unsupervised network that compares these blockwise motion vectors with the current homography flow, learns masks that mark which motion agrees with the dominant plane, and uses the masked information both to fuse motion into the estimator and to focus the training loss. If the argument is right, video-enabled devices get improved image alignment at no extra bitrate and without labels, exactly in the conditions where appearance-based methods struggle: low light, low texture, fog, rain, snow, and moving foreground objects. On CA-unsup the method reports the best average point-matching error among the compared learning-based methods, 0.31 versus 0.34 for the previous best, and on the unseen GHOF test set it reports 1.21 versus 1.72.","feed_headline":"Discarded H.264 motion vectors cut homography error to a new low","feed_subtitle":"Codec motion vectors reveal the dominant plane, beating prior unsupervised methods on CA-unsup and GHOF.","key_machinery":"The load-bearing mechanism is a pair of learned masks built by comparing two motion fields: the blockwise motion-vector flow $V^i_{ab}$ extracted from the H.264 bitstream and the current homography flow $H^i_{ab}$. In the Mask-Guided Fusion (MGF) module, a mask generator $G_m$ outputs a motion-rejection mask $M^i_m$ whose confidence values are modeled with conditionally independent Laplace distributions, and a fusion network predicts a residual homography added to $H^i_{ab}$. The Mask-Guided Homography Estimation (MGHE) module applies an adjusted mask $\\hat{M}^{i-1}_m = M^{i-1}_m(1-\\alpha)+\\alpha$ to the warped source and target features before a Swin transformer refinement, so early noisy estimates cannot zero out the features. Finally, the Enhanced Motion Mask $M_e = G_c(F'_a,F_b)\\cdot G_m(H_{ab},V_{ab})$ multiplies a coplanarity-aware feature mask with the motion-rejection mask and weights the unsupervised loss, trained with a negative log-likelihood term that keeps the mask from collapsing to all zeros.","core_discovery":"The paper's central claim is that motion vectors already present in coded video, which are ordinarily discarded after decoding, constitute an effective and entirely free prior for unsupervised homography estimation, and that a network can learn to use them by masking. CodingHomo predicts a homography from reconstructed H.264 frame pairs plus their motion-vector flows; the mask-guided fusion and mask-guided estimation modules suppress foreground and parallax motion while the enhanced motion mask concentrates the loss on the dominant plane. The paper reports that this reaches an average point-matching error of 0.31 on CA-unsup, a reduction from 0.34 for the previous best method, and 1.21 on GHOF, a reduction from 1.72, including generalization to fog, rain, and snow scenes not present in training.","pith_inferences":["Editorial extension: the same mask-guided fusion pattern could be applied to other decode-side priors, such as residual energy, block partition structure, or reference-frame indexes, to distinguish static background from dynamic objects; the paper does not explore these.","Editorial boundary: because H.264 motion search is typically capped around a 16-pixel window, the method is inherently a small-baseline estimator, and feature-based methods may remain necessary for large displacements unless codec search ranges are enlarged.","Editorial test: swapping the H.264 MVs for optical flow computed from the decoded frames would show whether the gain comes specifically from the codec prior or from any dense motion signal; the paper's mechanism does not logically require the MVs to originate from a codec."],"forward_implications":["Because the motion vectors are extracted from the bitstream during normal decoding, the method adds no bitrate cost and no new labeling; any pipeline that already decodes H.264 video can obtain the prior for free.","The reported generalization to fog, rain, and snow on GHOF, categories absent from training, indicates that the MV prior transfers across appearance domains better than photometric or feature losses alone.","The ablation numbers attribute most of the gain to the learned masks: removing the motion-rejection mask raises CA-unsup error from 0.31 to 0.36, and removing the mask from the loss raises it to 0.46, so the mask mechanism, not the raw MVs, is what carries the improvement.","The coarse-to-fine estimator inherits a transformer pyramid, but the MGHE module lets the mask prune features before each refinement stage, which is what the paper credits for handling large foregrounds and multiple planes.","The approach is stated to extend naturally to HEVC/H.265 and VVC/H.266, since MV extraction is a standard decoding step; the H.264 choice is made because the CA-unsup dataset was encoded in that format."],"supporting_citations":[{"why":"Supplies the CA-unsup dataset of H.264-encoded real video pairs with ground-truth matches, the feature projector, and the triplet-loss baseline that this method builds on.","marker":"[25]"},{"why":"HomoGAN contributes the multi-scale CNN feature pyramid, the Swin-transformer coarse-to-fine estimation block, and the coplanarity-aware mask idea that the enhanced motion mask adapts.","marker":"[34]"},{"why":"RealSH is the previous best learning-based baseline (0.34 PME on CA-unsup) that CodingHomo must beat, and its realistic dataset generation defines the supervised comparison.","marker":"[30]"},{"why":"GHOF provides the cross-dataset generalization test with fog, rain, and snow frames plus sparse ground-truth correspondences used for the 1.21 PME result.","marker":"[26]"},{"why":"PDC-Net supplies the probabilistic dense-correspondence formulation, modeled as conditionally independent Laplace distributions, that the motion-rejection mask adopts for confidence estimation.","marker":"[63]"},{"why":"CodingFlow demonstrates that motion vectors from video coding can serve as a motion prior for a downstream vision task, motivating the use of MVs here.","marker":"[35]"},{"why":"The H.264/AVC standard defines the macroblock inter prediction and motion vector structure from which the paper extracts MVs during decoding.","marker":"[36]"}],"fun_headline_variants":[],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The masks only help if the codec's motion vectors and the current homography agree about the dominant plane; when the initial homography is far wrong or the true motion is larger than the codec's short search window, the prior stops being informative and the method has no backup signal.","fun_headline_variants_meta":{"error":"Client error '402 Payment Required' for url 'https://api.deepseek.com/chat/completions'\nFor more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/402"},"cache_creation_input_tokens":0},"created_at":"2026-08-16T12:36:22.216366+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Replace the H.264 motion-vector input with a scrambled or all-zero flow on the CA-unsup test set and compare average point-matching error: if it remains near 0.31 the motion prior is not load-bearing, and if it jumps toward the 0.36-0.46 range seen in the no-mask ablations, the prior is confirmed as the source of the gain.","supporting_citations":[{"cited_title":"Content-aware unsupervised deep homography estimation,","cited_arxiv_id":null,"evidence_quote":"Supplies the CA-unsup dataset of H.264-encoded real video pairs with ground-truth matches, the feature projector, and the triplet-loss baseline that this method builds on."},{"cited_title":"Unsupervised homography estimation with coplanarity-aware gan,","cited_arxiv_id":null,"evidence_quote":"HomoGAN contributes the multi-scale CNN feature pyramid, the Swin-transformer coarse-to-fine estimation block, and the coplanarity-aware mask idea that the enhanced motion mask adapts."},{"cited_title":"Supervised homography learning with realistic dataset generation,","cited_arxiv_id":null,"evidence_quote":"RealSH is the previous best learning-based baseline (0.34 PME on CA-unsup) that CodingHomo must beat, and its realistic dataset generation defines the supervised comparison."},{"cited_title":"Gyroflow+: Gyroscope-guided unsupervised deep homography and optical flow learning,","cited_arxiv_id":null,"evidence_quote":"GHOF provides the cross-dataset generalization test with fog, rain, and snow frames plus sparse ground-truth correspondences used for the 1.21 PME result."},{"cited_title":"Pdc-net+: En- hanced probabilistic dense correspondence network,","cited_arxiv_id":null,"evidence_quote":"PDC-Net supplies the probabilistic dense-correspondence formulation, modeled as conditionally independent Laplace distributions, that the motion-rejection mask adopts for confidence estimation."},{"cited_title":"Overview of the h. 264/avc video coding standard,","cited_arxiv_id":null,"evidence_quote":"The H.264/AVC standard defines the macroblock inter prediction and motion vector structure from which the paper extracts MVs during decoding."}],"review_version":1}