DaD: Distilled Reinforcement Learning for Diverse Keypoint Detection
read the original abstract
Keypoints are what enable Structure-from-Motion (SfM) systems to scale to thousands of images. However, designing a keypoint detection objective is a non-trivial task, as SfM is non-differentiable. Typically, an auxiliary objective involving a descriptor is optimized. This however induces a dependency on the descriptor, which is undesirable. In this paper we propose a fully self-supervised and descriptor-free objective for keypoint detection, through reinforcement learning. To ensure training does not degenerate, we leverage a balanced top-K sampling strategy. While this already produces competitive models, we find that two qualitatively different types of detectors emerge, which are only able to detect light and dark keypoints respectively. To remedy this, we train a third detector, DaD, that optimizes the Kullback-Leibler divergence of the pointwise maximum of both light and dark detectors. Our approach significantly improve upon SotA across a range of benchmarks. Code and model weights are publicly available at https://github.com/parskatt/dad
This paper has not been read by Pith yet.
Forward citations
Cited by 5 Pith papers
-
From Pairs to Sequences: Track-Aware Policy Gradients for Keypoint Detection
TraqPoint uses a track-aware reward and policy gradients in RL to train keypoint detectors for better trackability across sequences, outperforming prior methods on matching and reconstruction benchmarks.
-
OrthoTrack: Continuous 6-DoF UAV Trajectory Estimation Anchored in Public Orthophotos
OrthoTrack is a training-free system for continuous 6-DoF UAV pose estimation anchored in public orthophotos and surface models, with a new MovingDrone benchmark dataset.
-
OrthoTrack: Continuous 6-DoF UAV Trajectory Estimation Anchored in Public Orthophotos
OrthoTrack is a training-free system for continuous metric 6-DoF UAV pose estimation anchored in public orthophotos and surface models, with a new MovingDrone benchmark dataset.
-
LoMa: Local Feature Matching Revisited
Scaling data, model size, and compute for local feature matching produces large performance gains on challenging benchmarks and a new manually annotated HardMatch dataset.
-
Who Handles Orientation? Investigating Invariance in Feature Matching
Learning rotation invariance in descriptors matches the performance of matcher-level invariance but allows earlier invariance, faster matchers, and no loss in upright performance when trained at scale.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.