REVIEW 3 major objections 6 minor 40 references
Dense Match Summarization for Faster Two-view Estimation
T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Dense matches can be compressed into a handful of summarized correspondences without losing pose accuracy.
desk verdict A genuinely useful engineering trick for speeding up RANSAC on dense matches, with a runtime-accounting flaw that needs fixing before publication. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the reduced measurement matrix M_k, a 9x9 matrix computed per cluster via the Cholesky factorization A_k^T A_k = M_k^T M_k, where A_k stacks the Kronecker-product constraint rows of the matches in the cluster. It collapses the cluster's geometric information into a single matrix, so evaluating the proxy residual f_approx(E) = sum_k min( ||M_k e||^2 / $\alpha$(E; C_k), |C_k| $tau^{2}$ ) costs O(K) rather than O(N). The denominator $\alpha$(E; C_k) uses the representative match's coordinates to approximate the Sampson denominator for every match in the cluster.
What would settle it
Take a weakly textured indoor image pair, cluster its dense matches, label every match as inlier or outlier under the ground-truth essential matrix, and count clusters that contain both labels; if a substantial fraction of clusters are mixed, the per-cluster inlier ratio from the proxy residual will diverge from the per-match inlier ratio and the accuracy-equivalence result should break down on those pairs.
Extended reading notes
Core claim
The central claim is that robust two-view relative pose estimation from dense correspondences can be reformulated so that the cost function operates on roughly one percent of the matches, with each summarized match carrying a 9x9 reduced measurement matrix. For a cluster of matches, the sum of Sampson residuals is approximated by a single quadratic term, and the all-inlier-or-all-outlier assumption turns the truncated cluster cost into the minimum of that quadratic and a constant. On MegaDepth and ScanNet with several dense matchers, the paper reports pose accuracy close to the dense baseline at a fraction of the runtime, and it shows the same behavior for fundamental matrix estimation on a wide-baseline benchmark.
Load-bearing premise
The whole proxy cost rests on assuming each cluster of matches is either entirely inliers or entirely outliers, so the truncated cluster cost can be replaced by the minimum of the cluster's full residual sum and the constant penalty.
Editorial extensions
If this is right
- Robust two-view estimation with dense matches becomes feasible in real-time settings, since the RANSAC scoring and refinement cost drops by one to two orders of magnitude.
- The summarization is matcher-agnostic, applying equally to matches from DKM, RoMA, ASpanFormer, and MASt3R, so future dense matchers can inherit the speedup without modification.
- The same proxy-residual construction transfers to fundamental matrix estimation, giving comparable speedups on wide-baseline imagery where intrinsics are unknown.
- Because each cluster's constraints are fully contained in one 9x9 matrix plus a single correspondence, the representation also offers a compact storage form for dense match sets.
- The accuracy-runtime trade-off is controlled by one hyperparameter, the number of clusters K, allowing a smooth transition from the fastest sparse scoring to near-dense accuracy.
Reading between the lines
- The reduced measurement matrix construction is not specific to two-view geometry; the same Cholesky-based collapse could summarize constraints for homography, absolute pose, or other minimal problems where residuals are quadratic in the model parameters.
- The all-inlier-or-all-outlier cluster assumption predicts that the method's accuracy should degrade specifically in scenes where clusters straddle epipolar boundaries or depth discontinuities; counting such mixed clusters on real data would quantify the main failure mode.
- The paper's speedup is measured for the robust estimation stage only, so total runtime gains are smaller when the dense matcher itself dominates; combining this summarization with faster dense matching is a natural next step.
- The approach appears composable with RANSAC acceleration techniques that prune or reorder scoring, since it works on the scoring and refinement stages rather than on sampling.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a method for accelerating robust two-view relative pose estimation from dense correspondences. The dense set of N matches is clustered into K groups, each represented by a single representative match; each cluster is also summarized by a 9x9 matrix M_k obtained from the reduced measurement matrix, giving a proxy residual f_approx(E) in Eq. (12). This proxy is used for MSAC scoring and/or local refinement inside a PoseLib-based LO-RANSAC pipeline. Experiments on MegaDepth-1500, ScanNet-1500, and WxBS with DKM, RoMA, ASpanFormer, and MASt3R report pose accuracy comparable to the dense baseline at 10-100x lower robust-estimation runtime.
Significance. If the reported runtime includes all steps introduced by the method and the mixed-cluster approximation is validated on the actual optimization trajectory, this is a practically valuable contribution. The derivation is clean, the 9x9 matrix formulation is elegant, and the evaluation is extensive: three dense matchers, two standard benchmarks, a wide-baseline dataset, ablations over clustering methods and K, and 10 seeds per configuration. The method is orthogonal to many prior RANSAC acceleration techniques and could make dense-match robust estimation practical at millisecond cost. The paper also states its limitations clearly, notably that only the robust estimation step is accelerated and not the dense matcher itself.
major comments (3)
- [§4.1, Table 1, Table 3] The speedup numbers in Table 3 and Table 4 appear to exclude the clustering time. Table 1 reports K-means 4D clustering at 1.22 ms and RANSAC at 1.44 ms, while Table 3 reports CCA with RT = 1.5 ms and a 45.2x speedup over the 66.0 ms dense baseline. If the 1.5 ms is RANSAC-only and excludes the 1.22 ms clustering step, the total method time is approximately 2.7 ms, giving about a 24x speedup rather than 45.2x; the same issue affects Table 4 and Table 5. Since the dense baseline does not include a clustering step, a fair comparison must report end-to-end robust estimation time including clustering, or explicitly state that the speedup is internal to RANSAC and give clustering overhead separately in the main tables.
- [§3.2, Eq. (6); §4.3, Fig. 5 and Table 2] The simplification from Eq. (5) to Eq. (6) is exact only if every cluster is all-inlier or all-outlier. For mixed clusters, min(sum_i r_i, |C| tau^2) is not generally equal to sum_i min(r_i, tau^2), and the resulting bias can affect model ranking and refinement. The validation in Section 4.3 evaluates the denominator approximation at the ground-truth essential matrix only, not the full proxy cost over hypotheses visited by RANSAC, and Table 2 reports inlier ratios at E_gt rather than the error of f_approx relative to the dense MSAC cost. I ask the authors to quantify the frequency of mixed clusters using ground-truth inlier labels at the RANSAC threshold, and to report the distribution of f_approx(E) - f(E) over sampled and refined essential matrices, or the rank correlation between the two costs on a sample of hypotheses.
- [§4.5, Table 4] The accuracy differences between CCA/CCC and the dense baseline are small in absolute AUC terms, but the central claim of 'comparable accuracy' would be strengthened by a paired statistical analysis across the 1500 image pairs rather than only mean AUC and standard deviation over 10 seeds. For example, reporting the fraction of image pairs where the summarization changes the pose error by more than 1 degree would show whether the small mean degradation is driven by a few outliers or by a systematic small loss.
minor comments (6)
- [Figure 3 caption] The caption lists the clustering methods as '(2D, 3D, 9D)' but the text and figure describe 2D, 4D, and 9D K-means; '3D' should be '4D'.
- [Conclusion, Section 5] The sentence 'we can choose such that N << 9K' appears to have the inequality reversed; with N = 10,000 and K = 128, 9K = 1,152, so the intended statement is 9K << N.
- [Table 2] The inlier ratio for the exact residual is per match while that for the approximate residual is per cluster; the table should define what an inlier cluster is and note that the two ratios are not directly comparable.
- [Section 4.3] The approximation-error evaluation uses K-means 9D while the main method uses K-means 4D; please justify that the 9D clustering is representative or repeat the evaluation with the main 4D clustering.
- [Section 4.1] The runtime setup is described only as 'a modern desktop CPU'; specifying the CPU model, number of threads, and compiler settings would improve reproducibility.
- [Section 4.2] The hyperparameter K = 128 is selected on MegaDepth-1500, which is also the test set for the ablation; this is a mild form of tuning, but a sentence acknowledging it and reporting sensitivity on a held-out split would make the evaluation cleaner.
Circularity Check
No significant circularity: f_approx is an explicit approximation of the Sampson/MSAC cost, not a refit of the target pose; self-citations are implementation-level only.
full rationale
The derivation of the proxy cost is self-contained. Equation (12) is obtained from the exact MSAC cost (3) by three explicit steps: the all-inlier/all-outlier rewrite (Eqs. 5-6), the denominator replacement using the representative match (Eq. 8), and the reduced measurement matrix identity (Eq. 11). All quantities entering f_approx (clusters C_k, representative matches (c_k, cbar_k), matrices M_k, and alpha(E;C_k)) are constructed from the dense input matches alone, not from the target essential matrix or from pose-accuracy labels. The approximation is therefore not equivalent to its input by construction; its quality is an empirical question, and the paper tests it against ground-truth poses on MegaDepth-1500, ScanNet-1500, and WxBS with three dense matchers. The only self-citation of note is PoseLib [23], which is used as the LO-RANSAC implementation framework; this is a software dependency, not a mathematical premise invoked to force the result. The selection of K=128 on MegaDepth-1500, which also appears in the final benchmark, is test-set hyperparameter tuning and a reproducibility caveat, but it is not circular reasoning because the proxy residual itself is not fitted to target pose data.
Assumptions & free parameters
free parameters (1)
- K (number of clusters) =
128
assumptions (4)
- domain assumption Matches close to each other in the two images produce similar residuals for the relative pose cost function (eq. 3).
- ad hoc to paper Each cluster of matches is either entirely inlier or entirely outlier.
- ad hoc to paper The Sampson error denominator can be approximated using the representative match (c, cbar) instead of each match (x, xbar).
- standard math Cholesky factorization of A^T A exists and the reduced measurement matrix trick is exact.
Cite this review
Pith. "Pith review of Dense Match Summarization for Faster Two-view Estimation." pith.science (2026). https://pith.science/paper/OPXS2XDJ
@misc{pith2026250602893,
author = {Pith},
title = {Pith review of: Dense Match Summarization for Faster Two-view Estimation},
year = {2026},
howpublished = {\url{https://pith.science/paper/OPXS2XDJ}},
note = {Machine review of arXiv:2506.02893}
}
read the original abstract
In this paper, we speed up robust two-view relative pose from dense correspondences. Previous work has shown that dense matchers can significantly improve both accuracy and robustness in the resulting pose. However, the large number of matches comes with a significantly increased runtime during robust estimation in RANSAC. To avoid this, we propose an efficient match summarization scheme which provides comparable accuracy to using the full set of dense matches, while having 10-100x faster runtime. We validate our approach on standard benchmark datasets together with multiple state-of-the-art dense matchers.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurelien Lucchi, Pascal Fua, and Sabine S ¨usstrunk. Slic superpix- els compared to state-of-the-art superpixel methods.IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(11):2274–2282, 2012. 5
work page 2012
-
[2]
Daniel Barath and G ´abor Valasek. Space-partitioning ransac. InEuropean Conference on Computer Vision (ECCV), 2022. 2
work page 2022
-
[3]
MAGSAC: marginalizing sample consensus
Daniel Barath, Jiri Matas, and Jana Noskova. MAGSAC: marginalizing sample consensus. InComputer Vision and Pattern Recognition (CVPR), 2019. 2
work page 2019
-
[4]
Two-view geometry scoring without correspondences
Axel Barroso-Laguna, Eric Brachmann, Victor Adrian Prisacariu, Gabriel J Brostow, and Daniyar Turmukhambe- tov. Two-view geometry scoring without correspondences. InComputer Vision and Pattern Recognition (CVPR), 2023. 2
work page 2023
-
[5]
Jacob Bentolila and Joseph M Francos. Conic epipolar con- straints from affine correspondences.Computer Vision and Image Understanding (CVIU), 2014. 2
work page 2014
-
[6]
Luca Cavalli, Daniel Barath, Marc Pollefeys, and Vik- tor Larsson. Consensus-adaptive ransac.arXiv preprint arXiv:2307.14030, 2023. 2
work page Pith review arXiv 2023
-
[7]
Hongkai Chen, Zixin Luo, Lei Zhou, Yurun Tian, Mingmin Zhen, Tian Fang, David McKinnon, Yanghai Tsin, and Long Quan. Aspanformer: Detector-free image matching with adaptive span transformer.European Conference on Com- puter Vision (ECCV), 2022. 2, 7
work page 2022
-
[8]
O. Chum and J. Matas. Matching with prosac - progressive sample consensus. InComputer Vision and Pattern Recogni- tion (CVPR), 2005. 2
work page 2005
Show all 40 references
-
[9]
Optimal randomized ransac
Ond ˇrej Chum and Ji ˇr´ı Matas. Optimal randomized ransac. IEEE Trans. Pattern Analysis and Machine Intelligence (PAMI), 2008. 2
2008
-
[10]
Locally op- timized ransac
Ond ˇrej Chum, Ji ˇr´ı Matas, and Josef Kittler. Locally op- timized ransac. InPattern Recognition, pages 236–243, Berlin, Heidelberg, 2003. Springer Berlin Heidelberg. 2
2003
-
[11]
Chang, Manolis Savva, Maciej Hal- ber, Thomas Funkhouser, and Matthias Nießner
Angela Dai, Angel X. Chang, Manolis Savva, Maciej Hal- ber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Computer Vision and Pattern Recognition (CVPR), 2017. 4
2017
-
[12]
Superpoint: Self-supervised interest point detection and description
Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabi- novich. Superpoint: Self-supervised interest point detection and description. InComputer Vision and Pattern Recognition Workshops (CVPRW), 2018. 2, 8
2018
-
[13]
The faiss library
Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazar´e, Maria Lomeli, Lucas Hosseini, and Herv´e J´egou. The faiss library. arXiv preprint arXiv:2401.08281, 2024. 5
2024 arXiv
-
[14]
D2- net: A trainable cnn for joint description and detection of local features
Mihai Dusmanu, Ignacio Rocco, Tomas Pajdla, Marc Polle- feys, Josef Sivic, Akihiko Torii, and Torsten Sattler. D2- net: A trainable cnn for joint description and detection of local features. InComputer Vision and Pattern Recognition (CVPR), 2019. 2
2019
-
[15]
DKM: Dense kernelized feature matching for geometry estimation
Johan Edstedt, Ioannis Athanasiadis, M ˚arten Wadenb ¨ack, and Michael Felsberg. DKM: Dense kernelized feature matching for geometry estimation. InComputer Vision and Pattern Recognition (CVPR), 2023. 1, 2, 4, 7
2023
-
[16]
RoMa: Robust Dense Feature Matching.Computer Vision and Pattern Recogni- tion (CVPR), 2024
Johan Edstedt, Qiyu Sun, Georg B ¨okman, M ˚arten Wadenb¨ack, and Michael Felsberg. RoMa: Robust Dense Feature Matching.Computer Vision and Pattern Recogni- tion (CVPR), 2024. 2, 7
2024
-
[17]
Affine correspon- dences between central cameras for rapid relative pose es- timation
Iv ´an Eichhardt and Dmitry Chetverikov. Affine correspon- dences between central cameras for rapid relative pose es- timation. InEuropean Conference on Computer Vision (ECCV), 2018. 2
2018
-
[18]
Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography.Communications of the ACM, 24(6):381–395, 1981
Martin A Fischler and Robert C Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography.Communications of the ACM, 24(6):381–395, 1981. 1, 2
1981
-
[19]
Fast glob- ally optimal surface normal estimation from an affine corre- spondence
Levente Hajder, Lajos L ´oczi, and Daniel Barath. Fast glob- ally optimal surface normal estimation from an affine corre- spondence. InInternational Conference on Computer Vision (ICCV), 2023. 2
2023
-
[20]
Cotr: Correspondence transformer for matching across images
Wei Jiang, Eduard Trulls, Jan Hosang, Andrea Tagliasac- chi, and Kwang Moo Yi. Cotr: Correspondence transformer for matching across images. InInternational Conference on Computer Vision (ICCV), 2021. 2
2021
-
[21]
Fast-slic.https://github.com/Algy/ fast-slic, 2019
Alchan Kim. Fast-slic.https://github.com/Algy/ fast-slic, 2019. 5
2019
-
[22]
Latent ransac
Simon Korman and Roee Litman. Latent ransac. InCom- puter Vision and Pattern Recognition (CVPR), 2018. 2
2018
-
[23]
PoseLib - Minimal Solvers for Camera Pose Estimation.https://github.com/ vlarsson/PoseLib, 2020
Viktor Larsson and contributors. PoseLib - Minimal Solvers for Camera Pose Estimation.https://github.com/ vlarsson/PoseLib, 2020. 4
2020
-
[24]
Fixing the locally optimized ransac–full experimental evaluation
Karel Lebeda, Jirı Matas, and Ondrej Chum. Fixing the locally optimized ransac–full experimental evaluation. In British Machine Vision Conference (BMVC), 2012. 2
2012
-
[25]
Ground- ing image matching in 3d with mast3r
Vincent Leroy, Yohann Cabon, and J´erˆome Revaud. Ground- ing image matching in 3d with mast3r. InEuropean Confer- ence on Computer Vision (ECCV), 2024. 8
2024
-
[26]
Megadepth: Learning single- view depth prediction from internet photos
Zhengqi Li and Noah Snavely. Megadepth: Learning single- view depth prediction from internet photos. InComputer Vision and Pattern Recognition (CVPR), 2018. 1, 4
2018
-
[27]
LightGlue: Local Feature Matching at Light Speed
Philipp Lindenberger, Paul-Edouard Sarlin, and Marc Polle- feys. LightGlue: Local Feature Matching at Light Speed. In International Conference on Computer Vision (ICCV), 2023. 2, 8
2023
-
[28]
Distinctive image features from scale- invariant keypoints.International Journal of Computer Vi- sion (IJCV), 60:91–110, 2004
David G Lowe. Distinctive image features from scale- invariant keypoints.International Journal of Computer Vi- sion (IJCV), 60:91–110, 2004. 2
2004
-
[29]
WxBS: Wide baseline stereo generalizations
Dmytro Mishkin, Jiri Matas, Michal Perdoch, and Karel Lenc. WxBS: Wide baseline stereo generalizations. In British Machine Vision Conference (BMVC), 2015. 4, 8
2015
-
[30]
Groupsac: Efficient consensus in the presence of groupings
Kai Ni, Hailin Jin, and Frank Dellaert. Groupsac: Efficient consensus in the presence of groupings. InInternational Conference on Computer Vision (ICCV), 2009. 2
2009
-
[31]
An efficient solution to the five-point relative pose problem.IEEE Trans
David Nist ´er. An efficient solution to the five-point relative pose problem.IEEE Trans. Pattern Analysis and Machine Intelligence (PAMI), 2004. 7
2004
-
[32]
Ac- curate motion estimation through random sample aggregated consensus.arXiv preprint arXiv:1701.05268, 2017
Martin Rais, Gabriele Facciolo, Enric Meinhardt-Llopis, Jean-Michel Morel, Antoni Buades, and Bartomeu Coll. Ac- curate motion estimation through random sample aggregated consensus.arXiv preprint arXiv:1701.05268, 2017. 2 9
2017 arXiv
-
[33]
Theory and practice of structure-from-motion using affine correspondences
Carolina Raposo and Joao P Barreto. Theory and practice of structure-from-motion using affine correspondences. In Computer Vision and Pattern Recognition (CVPR), 2016. 2
2016
-
[34]
A. L. Rodr ´ıguez, P. E. L ´opez-de Teruel, and A. Ruiz. Re- duced epipolar cost for accelerated incremental SfM. In Computer Vision and Pattern Recognition (CVPR), 2011. 4
2011
-
[35]
Superglue: Learning feature matching with graph neural networks
Paul-Edouard Sarlin, Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. Superglue: Learning feature matching with graph neural networks. InComputer Vision and Pattern Recognition (CVPR), 2020. 2, 4
2020
-
[36]
LoFTR: Detector-free local feature matching with transformers.Computer Vision and Pattern Recognition (CVPR), 2021
Jiaming Sun, Zehong Shen, Yuang Wang, Hujun Bao, and Xiaowei Zhou. LoFTR: Detector-free local feature matching with transformers.Computer Vision and Pattern Recognition (CVPR), 2021. 2, 4
2021
-
[37]
Torr and A
P.H.S. Torr and A. Zisserman. MLESAC: A New Robust Estimator with Application to Estimating Image Geometry. Computer Vision and Image Understanding (CVIU), 78(1): 138–156, 2000. 2
2000
-
[38]
P1ac: Revisiting absolute pose from a sin- gle affine correspondence
Jonathan Ventura, Zuzana Kukelova, Torsten Sattler, and D´aniel Bar ´ath. P1ac: Revisiting absolute pose from a sin- gle affine correspondence. InInternational Conference on Computer Vision (ICCV), 2023. 2
2023
-
[39]
Adaptive reorder- ing sampler with neurally guided magsac
Tong Wei, Jiri Matas, and Daniel Barath. Adaptive reorder- ing sampler with neurally guided magsac. InInternational Conference on Computer Vision (ICCV), 2023. 2 10 Dense Match Summarization for Faster Two-view Estimation Supplementary Material AUC Runtime (ms) Sampling Scorin...
2023
-
[40]
In addition to AUC@5 ◦, we also present AUC@10◦ and AUC@20◦
Additional Ablation Results In Table 6, we include some additional results on the ab- lation on RANSAC integration to what was presented in the main paper. In addition to AUC@5 ◦, we also present AUC@10◦ and AUC@20◦. In addition to the median es- timation runtime, we also pres...
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.