Consistent Point Matching

Gerardo Hermosillo Valadez; Halid Ziya Yerebakan

REVIEW 3 major objections 5 minor 13 references

Consistent Point Matching

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Adding a round-trip consistency check to point matching improves anatomical correspondence accuracy across CT and MRI pairs, surpassing a supervised deep-learning baseline on the Deep Lesion Tracking dataset without any training.

desk verdict The paper pairs a useful heuristic with an experimental design that confounds the consistency mechanism against a simultaneous descriptor upgrade, so the central attribution is unsupported. read the letter →

arxiv 2507.23609 v1 pith:O2LDT4S4 submitted 2025-07-31 cs.CV cs.DCcs.LG

Halid Ziya Yerebakan , Gerardo Hermosillo Valadez This is my paper · ORCID

classification cs.CVcs.DCcs.LG

keywords consistentpointmatchinground-tripconsistencyanatomicallongitudinalmedicalimageanalysislandmarklocalizationDeepLesionsparsesamplingdescriptortraining-free

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that matching anatomical points between pairs of medical images becomes more robust if the matching algorithm checks itself: after finding a candidate target location, it maps that location back and penalizes matches whose round-trip distance from the original query is large. The authors build this consistency heuristic into a hierarchical, sparse-sampling descriptor search called Point Matching, using nearby points to vote on the best target location. On the DeepLesion longitudinal CT tracking test set, the method reaches 0.892 sensitivity at 10 mm, above both plain Point Matching (0.855) and the published supervised UAE operating point (0.841). Similar improvements appear on an internal lung-lesion dataset, a multi-modal CT/MR follow-up dataset, and carina landmark localization. If the results hold, accurate navigation between scans would not require a learned model or training data, only a few seconds on a CPU.

What carries the argument

The central mechanism is the round-trip consistency check inside a multi-resolution sparse-sampling point matcher. The descriptor is built by sparse sampling of intensity values at offsets specified in millimeters, and the search proceeds hierarchically from a 16 mm step down to 1 mm. For each query, the algorithm maps the point to the target, maps the returned location back to the source, and uses the resulting distance $d_i$ to weight the vote of each nearby point as $\exp(-d_i/s_0)$ times a similarity combining mutual information and cosine similarity; the top five weighted votes are averaged to give the final estimated location. This consistency weighting is what carries the argument, because it converts an unverifiable one-way match into a self-checking match that can be compared across neighboring locations.

What would settle it

A concrete test would be to run Consistent Point Matching and plain Point Matching on pairs of CT scans with large, non-uniform deformation—for example, lung volumes at full inspiration versus full expiration, where the diaphragm moves much more than the apex—and check whether the consistency version loses its advantage or becomes worse than plain point matching as the local displacement gradient increases. If round-trip consistency does not predict match accuracy in such cases, the heuristic's claimed general robustness fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that incorporating consistency—defined as the distance between an original query point and its round-trip estimate after mapping to the target image and back—into the similarity search makes correspondence finding more reliable. The algorithm computes round-trip distances for the central query and twelve nearby points at radius 1.5 and 0.5 times the current search step, weights each candidate by $\exp(-d_i/s_0)$ times its similarity score, keeps the best five votes, and averages their positions. On the DeepLesion test set this yields 0.892@10mm, exceeding the 0.855 of the base Point Matching method and the 0.841 of the supervised UAE method; it also raises lung-lesion matching from 0.931 to 0.954 at 10 mm, multi-modal follow-up matching to 95.2%, and carina localization from 0.933 to 0.985. The authors present this as evidence that a simple, training-free consistency heuristic can replace learned embeddings for some high-precision anatomical navigation tasks.

Load-bearing premise

The method assumes that nearby points in the source image move to similarly nearby points in the target image, so that the displacements estimated from neighboring points are reliable guides to the true correspondence; if tissue moves non-uniformly between scans, the neighbor votes can pull the estimate away from the correct location.

Editorial extensions

If this is right

Anatomical navigation in longitudinal CT and MR studies can be performed on a standard CPU in about two seconds per point, without a GPU or a learned model.
On the DeepLesion tracking benchmark, adding the consistency heuristic lifts performance above the published supervised UAE operating point, suggesting that round-trip verification provides some of the benefit that semantic embeddings are trained to supply.
Landmark localization with a single atlas template reaches 0.985@10mm, approaching supervised detector performance, which would make atlas-based annotation practical for new sites without local training data.
The method generalizes across CT and MRI modalities and across lesion types, so the same implementation could serve several clinical tasks that currently require separate trained models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The gain may be partly a spatial-smoothing effect rather than pure self-consistency: sampling twelve nearby offsets and averaging the best five resembles robust estimation; an ablation with random offsets (or offsets without round-trip weighting) would separate these contributions.
Forward-backward consistency is a known tool in optical flow and stereo matching, and this paper suggests the same idea transfers to descriptor-based medical image correspondence; one could test whether it improves other hierarchical matching schemes.
Since the round-trip check is independent of the descriptor, it could be applied as a post-processing re-ranking step to any candidate correspondence method, potentially capturing much of the gain without the full multi-neighbor search overhead.
The claimed failure mode near lesions or boundaries, where displacement is non-smooth, suggests a confidence map: if the dispersion of the top-five votes is large, the method could flag low-confidence matches for human review.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know about this one before reading: the abstract says consistency improves robustness, but the method section also adds three orthogonal sampling planes and an 80 mm grid to the descriptor at the same time. The experiments compare that combined change against the old point-matching baseline. There is no ablation that isolates consistency from the descriptor upgrade, so the observed gains (0.855 to 0.892@10mm on DeepLesion; mean distance 5.90 to 4.65 mm) could come entirely from the richer descriptor. That is a load-bearing flaw, and the stress-test note gets it right. What is genuinely new here is the specific combination: round-trip consistency from UAE, applied to a training-free hierarchical search, with multi-neighbor voting, top-5 aggregation, and the orthogonal-plane descriptor extension. That combination is not in prior work, and the authors are honest about borrowing the consistency idea from Bai et al. The empirical scope is decent; four datasets, CT and MRI, plus landmark localization; and the DeepLesion test set is used without training, which is the right way to evaluate a training-free method. The CPU speed claims are relevant and credible. The soft spots beyond the confound: no confidence intervals or significance tests anywhere, so the headline superiority over UAE rests on a single operating point (0.841 vs. 0.892) with no error bars. No code or public internal data makes verification hard. The local-displacement smoothness assumption for neighbor voting is worth noting, but it is a modeling limitation rather than a design flaw. The paper is worth taking seriously as a practical heuristic contribution, but not in its current form. The central claim needs an ablation: new descriptor with and without the consistency loop, ideally with multiple seeds or bootstrap intervals. I would send it to peer review because the method is plausible and the public benchmark is relevant, but I would insist on that ablation before accepting. If the improvement vanishes without consistency, the paper becomes a descriptor-tuning exercise; if it holds, the heuristic is a useful component. The authors seem capable and honest about their prior work, so the confound is likely a mistake rather than a dodge, but it must be fixed.

Referee Report

3 major / 5 minor

Summary. The manuscript extends the authors' earlier Point Matching method by adding a round-trip consistency heuristic: candidate correspondences are scored by how close their reverse mapping returns to the original query point, and votes from nearby points are aggregated. The proposed Consistent Point Matching is evaluated on four datasets: the public DeepLesion tracking test set, an internal lung-lesion CT dataset, a multi-modal CT/MR follow-up dataset, and a carina landmark-localization task. The paper reports consistent improvements over plain Point Matching (e.g., 0.855 to 0.892 at 10mm on DeepLesion, mean distance 5.90 to 4.65 mm) and claims state-of-the-art performance against the published UAE operating point, without requiring training or a learned model.

Significance. If the claimed improvement is real, this is a practically valuable result: a training-free, CPU-only matching heuristic that improves robustness on diverse modalities and outperforms a supervised state-of-the-art method on a public benchmark. The use of the DeepLesion test set without training is a strength, as is the breadth of the evaluation across four datasets. However, the central attribution of the improvement to the consistency heuristic is currently undermined by a simultaneous descriptor upgrade, and the state-of-the-art claim rests on a single published operating point without uncertainty quantification. These issues are fixable but require additional experiments or careful re-framing.

major comments (3)

[Methods — Consistent Point Matching] The reported comparison conflates the consistency heuristic with a descriptor upgrade. The paragraph after Algorithm 1 states: 'In addition to the algorithmic change, we have improved the descriptor definition by adding three orthogonal planes with a resolution of 6 mm using a 7x7 2D grid and an 80 mm 3D grid.' The Results section and Table 1 compare 'Point Matching' with 'Consistent Point Matching' without specifying which descriptor each condition uses. If the 'Point Matching' baseline is the original method from reference [1] with the original descriptor, then the observed gains (0.855 to 0.892 at 10mm on DeepLesion; mean distance 5.90 to 4.65 mm in Table 1) could be caused entirely by the richer descriptor, independent of the consistency heuristic. The authors should report an ablation using the new descriptor both with and without consistency, or explicitly state that the baseline already includes the new descriptor.
[Results — Deep Lesion Tracking] The state-of-the-art claim is supported only by a single published operating point for UAE (0.841@10mm) and no confidence intervals or significance tests are reported for the 0.892@10mm result, the 0.855@10mm baseline, or any of the FROC curves. The sentence 'Since the other methods are compared in this prior work we only compared to UAE' explains the choice of comparator but does not address statistical uncertainty. Because the DeepLesion test set is public and fixed, bootstrap or per-lesion confidence intervals would be straightforward and would substantially strengthen the central claim. As it stands, the conclusion that the consistency heuristic 'surpasses state-of-the-art results' is not statistically supported.
[Methods — Consistency and neighbor voting] The method's core modeling assumption is stated just before Algorithm 1: 'Assuming that nearby points have similar offsets in their corresponding positions, we can estimate the required displacement to find the target point.' This assumption is load-bearing because the algorithm averages votes from nearby points and weights them by consistency distance; when the assumption fails, for example near lesions, organ boundaries, or under large non-uniform deformation, the aggregation could degrade accuracy instead of improving it. The paper does not provide any diagnostic of displacement-field smoothness or a discussion of known failure modes. Since the abstract claims that consistency 'improves robustness,' the evaluation should demonstrate robustness in conditions where local smoothness is challenged, or at least explicitly bound the claim to settings where the assumption plausibly holds.

minor comments (5)

[Table 1 and Results — Deep Lesion Tracking] The reported timings are inconsistent: the text reports 1.31 s per match for consistent point matching and 0.16 s for point matching on DeepLesion, while Table 1 gives 1.06 s and 0.12 s for the same experiment. Please reconcile the numbers and specify the hardware and batching configuration.
[Algorithm 1] The algorithm uses the variable 'center' before it has been assigned at the first iteration. The text says the first search level covers the whole image space, but the pseudocode does not define the initial center. Clarify the initialization.
[Algorithm 1 and Table 1] The text says 'we do not discard estimates; instead, we retain the best five,' but Algorithm 1 takes the top-5 of the weighted scores, which discards 8 of 13 estimates. Also, for the 3-point variant in Table 1, the top-5 selection is undefined. Specify how the number of retained estimates is set for each variant and reconcile the wording.
[Results — Speed Precision Trade off] The parenthetical numbers in Table 1 (3, 7, 13) are not explained in the table or surrounding text. The text mentions 'three points include only stepsize /2 for laterality offsets, while seven points include six neighbors,' but the mapping between these descriptions and the column values should be stated explicitly.
[Methods — Descriptor definition] The descriptor upgrade is described only in one sentence. For reproducibility, provide the complete descriptor construction: how the three orthogonal planes are combined with the existing 7x7x7 grids, how the 6 mm and 80 mm grids are used, and how the descriptor is normalized or concatenated before computing mutual information and cosine similarity.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the consistency heuristic is an empirical heuristic evaluated on external ground truth; self-citations and the descriptor confound do not make the claim definitional.

full rationale

The paper is an empirical evaluation, not a derivation: the consistency heuristic (round-trip distance) is defined independently of the annotated ground truth, and the main comparison is against external DeepLesion tracking annotations and other labeled datasets. No parameter in Algorithm 1 is fitted to the evaluation labels; s0, neighbor offsets, and top-5 aggregation are stated as fixed implementation choices. The method builds on the authors' prior Point Matching (self-citations [1], [8]) and borrows the consistency idea from Bai et al.'s UAE, but this is normal method building: the prior descriptors and the UAE heuristic are external or previously published, and the claimed improvement is measured on data, not implied by the definitions. The simultaneous descriptor upgrade (three orthogonal planes at 6 mm and an 80 mm grid) means the ablation does not isolate the consistency heuristic, and the number of consistency points was selected using the DeepLesion test set; both are experimental validity concerns, but neither makes the central claim true by construction or reduces a 'prediction' to a fitted input. No circular step satisfying the quoted-reduction requirement was found.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on an empirical heuristic rather than a derivation. It inherits assumptions from prior Point Matching work and adds a smoothness assumption about nearby displacements. Several hand-chosen constants shape the algorithm, and none are accompanied by sensitivity analyses. No new physical entities are introduced.

free parameters (5)

s0 (gaussian scale for consistency weight) = 16 mm
Hand-chosen constant in Algorithm 1; controls how strongly round-trip distance penalizes candidate similarity.
Neighbor offset factors = 0.5s and 1.5s along each axis
Hand-selected offsets define the voting neighborhood; no sensitivity analysis is reported.
Top-k retained estimates = 5
The best five weighted neighbor estimates are averaged; this aggregation count is fixed without ablation.
Descriptor sampling grids = 7x7x7 at 8, 20, 48, 128 mm; new 7x7 2D at 6 mm and 80 mm 3D
Hand-designed descriptor configuration that affects discriminability; inherited and extended from prior Point Matching.
Hierarchical levels and step sizes = 5 levels, 16 mm down to 1 mm
Fixed multi-resolution schedule inherited from Point Matching; no ablation is shown in this paper.

assumptions (4)

domain assumption Nearby points have similar offsets in their corresponding positions.
Explicitly assumed before the voting step in Methods; if false, neighbor estimates would not converge to the target point.
domain assumption Round-trip consistency distance is a reliable proxy for correspondence accuracy.
The core heuristic used to weight candidate matches; validated empirically but not derived from first principles.
domain assumption Medical images in the tested datasets contain consistent anatomical structure and reliable reference-frame metadata.
Needed for millimeter-based sparse sampling descriptors and for single-template landmark transfer in the carina experiment.
domain assumption Sparse intensity sampling at predefined offsets captures sufficient anatomical information for matching.
Inherited from prior Point Matching work and not re-derived in this paper.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Consistent Point Matching." pith.science (2026). https://pith.science/paper/O2LDT4S4

@misc{pith2026250723609,
  author       = {Pith},
  title        = {Pith review of: Consistent Point Matching},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/O2LDT4S4}},
  note         = {Machine review of arXiv:2507.23609}
}

read the original abstract

This study demonstrates that incorporating a consistency heuristic into the point-matching algorithm \cite{yerebakan2023hierarchical} improves robustness in matching anatomical locations across pairs of medical images. We validated our approach on diverse longitudinal internal and public datasets spanning CT and MRI modalities. Notably, it surpasses state-of-the-art results on the Deep Lesion Tracking dataset. Additionally, we show that the method effectively addresses landmark localization. The algorithm operates efficiently on standard CPU hardware and allows configurable trade-offs between speed and robustness. The method enables high-precision navigation between medical images without requiring a machine learning model or training data.

Figures

Figures reproduced from arXiv: 2507.23609 by the authors.

**Figure 1.** Sparse Sampling for Point Matching1 The sampler can be easily scaled to perform matching at both fine and coarse resolutions. Offsets defined in millimeters can be multiplied by a scaling factor to enable zoom-in functionality and adjust sampling memory indexes. The Point Matching method specifically employs five hierarchical levels. At each level, the method iterates over a grid of candidate locations in the target… view at source ↗

**Figure 2.** Consistency is distance between original query and round trip estimate We compute the consistencies for multiple nearby locations and incorporate them into the similarity function. Assuming that nearby points have similar offsets in their corresponding positions, we can estimate the required displacement to find the target point. In our implementation, nearby points are selected as six neighbors within a radius of 1… view at source ↗

**Figure 3.** Longitudinal Matching Performance Multi-Modal Study We additionally evaluated our method on an in-house study dataset containing multi-time-point CT and MR modalities. This dataset includes aortic aneurysms, intracranial aneurysms (ICA), enlarged lymph nodes, kidney lesions, meningioma, and pulmonary nodule pathologies with a total of 339 pairs of intra-modality matches. In this study, the annotations were provided … view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Localization performance on different tasks It is certainly easy to scale to multiple templates for increased robustness. However, our experimental results indicate that even with a single template, landmark localization performance approaches the level of supervised l…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

13 extracted references · 10 canonical work pages

[1]

author Yerebakan, H. Z. , author Shinagawa, Y. , author Ranganath, M. , author Allen-Raffl, S. & author Valadez, G. H. title A hierarchical descriptor framework for on-the-fly anatomical location matching between longitudinal studies . In booktitle International Conference on Medical Image Computing and Computer-Assisted Intervention MTSAIL & LEAF Worksho...

work page 2023
[2]

SAM++: Enhancing Anatomic Matching using Semantic Information and Structural Inference

author Bai, X. & author Xia, Y. journal title Sam++: Enhancing anatomic matching using semantic information and structural inference . arXiv preprint arXiv:2306.13988 ( year 2023 )

work page Pith review arXiv 2023
[3]

author Vizitiu, A. et al. title Multi-scale self-supervised learning for longitudinal lesion tracking with optional supervision . In booktitle International Conference on Medical Image Computing and Computer-Assisted Intervention , pages 573--582 ( organization Springer , year 2023 )

work page 2023
[4]

author Cai, J. et al. title Deep lesion tracker: monitoring lesions in 4d longitudinal imaging studies . In booktitle Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 15159--15169 ( year 2021 )

work page 2021
[5]

author Bai, X. et al. journal title Uae: Universal anatomical embedding on multi-modality medical images . arXiv preprint arXiv:2311.15111 ( year 2023 )

work page Pith review arXiv 2023
[6]

, author Wang, X

author Yan, K. , author Wang, X. , author Lu, L. & author Summers, R. M. journal title Deeplesion: automated mining of large-scale lesion annotations and universal lesion detection with deep learning . Journal of medical imaging volume 5 , pages 036501 ( year 2018 )

work page 2018
[7]

author Codella, N. C. et al. journal title Medimageinsight: An open-source embedding model for general domain medical imaging . arXiv preprint arXiv:2410.06542 ( year 2024 )

arXiv 2024
[8]

author Weikert, T. et al. journal title Reduction in radiologist interpretation time of serial ct and mr imaging findings with deep learning identification of relevant priors, series and finding locations . Academic Radiology volume 30 , pages 2269--2279 ( year 2023 )

work page 2023

Show all 13 references

[9]

author Yerebakan, H. Z. , author Shinagawa, Y. & author Valadez, G. H. title Real time multi organ classification on computed tomography images . In booktitle MICCAI Workshop on Data Engineering in Medical Imaging , pages 1--10 ( organization Springer , year 2024 )

2024
[10]

author Yan, K. et al. journal title Sam: Self-supervised learning of pixel-wise anatomical embeddings in radiological images . IEEE Transactions on Medical Imaging volume 41 , pages 2658--2669 ( year 2022 )

2022
[11]

, author Sethi, D

author Jena, R. , author Sethi, D. , author Chaudhari, P. & author Gee, J. journal title Deep learning in medical image registration: Magic or mirage? Advances in Neural Information Processing Systems volume 37 , pages 108331--108353 ( year 2025 )

2025
[12]

, " * write output.state after.block = add.period write newline

ENTRY address archive author booktitle chapter edition editor eprint howpublished institution journal key month note number organization pages publisher school series title type url doi volume year label INTEGERS output.state before.all mid.sentence after.sentence after.block ...
[13]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

author Yerebakan, H. Z. , author Shinagawa, Y. , author Ranganath, M. , author Allen-Raffl, S. & author Valadez, G. H. title A hierarchical descriptor framework for on-the-fly anatomical location matching between longitudinal studies . In booktitle International Conference on Medical Image Computing and Computer-Assisted Intervention MTSAIL & LEAF Worksho...

work page 2023

[2] [2]

SAM++: Enhancing Anatomic Matching using Semantic Information and Structural Inference

author Bai, X. & author Xia, Y. journal title Sam++: Enhancing anatomic matching using semantic information and structural inference . arXiv preprint arXiv:2306.13988 ( year 2023 )

work page Pith review arXiv 2023

[3] [3]

author Vizitiu, A. et al. title Multi-scale self-supervised learning for longitudinal lesion tracking with optional supervision . In booktitle International Conference on Medical Image Computing and Computer-Assisted Intervention , pages 573--582 ( organization Springer , year 2023 )

work page 2023

[4] [4]

author Cai, J. et al. title Deep lesion tracker: monitoring lesions in 4d longitudinal imaging studies . In booktitle Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 15159--15169 ( year 2021 )

work page 2021

[5] [5]

author Bai, X. et al. journal title Uae: Universal anatomical embedding on multi-modality medical images . arXiv preprint arXiv:2311.15111 ( year 2023 )

work page Pith review arXiv 2023

[6] [6]

, author Wang, X

author Yan, K. , author Wang, X. , author Lu, L. & author Summers, R. M. journal title Deeplesion: automated mining of large-scale lesion annotations and universal lesion detection with deep learning . Journal of medical imaging volume 5 , pages 036501 ( year 2018 )

work page 2018

[7] [7]

author Codella, N. C. et al. journal title Medimageinsight: An open-source embedding model for general domain medical imaging . arXiv preprint arXiv:2410.06542 ( year 2024 )

arXiv 2024

[8] [8]

author Weikert, T. et al. journal title Reduction in radiologist interpretation time of serial ct and mr imaging findings with deep learning identification of relevant priors, series and finding locations . Academic Radiology volume 30 , pages 2269--2279 ( year 2023 )

work page 2023