Pith. sign in

REVIEW 4 minor 18 references

metric-learn: Metric Learning Algorithms in Python

T0 review · 0 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read metric-learn is a Python package that puts ten metric learning algorithms behind one scikit-learn-compatible interface, so distance learners can be cross-validated, pipelined, and compared like any other estimator.

desk verdict A clean, honest software paper for a genuinely useful package; no new science, but it doesn't claim any, and it should be published. read the letter →

arxiv 1908.04710 v3 pith:MQELVSAE submitted 2019-08-13 cs.LG stat.ML

William de Vazelhes , CJ Carey , Yuan Tang , Nathalie Vauquier , Aurélien Bellet This is my paper · ORCID
classification cs.LGstat.ML
keywords metriclearningMahalanobisdistancePythonpackagescikit-learnweaklysupervisednearestneighborsopensourcesoftware
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

metric-learn is an open source Python package that implements ten popular distance metric learning algorithms under a single API. It unifies supervised learners, pair learners, triplet learners, and quadruplet learners, all of which learn a Mahalanobis distance, meaning a Euclidean distance after a learned linear transformation. Because the interface is compatible with scikit-learn, users can run cross-validation, model selection, and pipelining with other estimators. The paper's contribution is primarily engineering: a well-tested, MIT-licensed package that makes metric learning algorithms easy to compare and deploy. A sympathetic reader would care because metric learning is useful for retrieval, nearest-neighbor classification, clustering, and dimensionality reduction, and previous implementations were scattered across incompatible Matlab and R code.

What carries the argument

The machinery is the Mahalanobis distance family combined with a unified estimator API. All ten algorithms optimize a distance of the form $D_L(x,x') = \sqrt{(Lx-Lx')^\top(Lx-Lx')}$, so the learned object is always a linear transformation $L$. The package expresses this through an abstract BaseMetricLearner class, inheriting from scikit-learn's BaseEstimator, and a MahalanobisMixin that provides get_metric, score_pairs, transform, and get_mahalanobis_matrix; supervised learners additionally inherit TransformerMixin so they can be pipelined with other estimators. That combination is what makes the algorithms interchangeable in model selection and evaluation.

What would settle it

Compare metric-learn's learned transformation matrices and distances against reference outputs from the original algorithm papers on a standard benchmark dataset; any material discrepancy would falsify the central claim.

Watch

Extended reading notes

Core claim

The central claim is that metric-learn delivers production-quality implementations of ten Mahalanobis metric learning algorithms in one scikit-learn-compatible package. These ten cover the main supervision types: supervised learners (NCA, LMNN, RCA, LFDA, MLKR), pair learners (MMC, ITML, SDML), one triplet learner (SCML), and one quadruplet learner (LSML). Each algorithm learns either a transformation matrix $L$ or directly a Mahalanobis matrix $M=L^\top L$, defining a distance $D_L(x,x')=\sqrt{(Lx-Lx')^\top(Lx-Lx')}$. The package's value is the unified interface: a single BaseMetricLearner class with get_metric and score_pairs methods, and support for scikit-learn's Pipeline, GridSearchCV, and cross-validation. As a project in the scikit-learn ecosystem, it is open to community contributions and enforced by tests and continuous integration.

Load-bearing premise

The package's claim to correctly implement ten algorithms rests on the assumption that its 97% test coverage actually checks numerical correctness of the learned metrics, not just that the code runs.

Editorial extensions

If this is right

  • Users can evaluate and compare all ten algorithms on the same data using scikit-learn's cross-validation and grid-search tools, without writing glue code.
  • A learned metric can be used directly in other scikit-learn estimators, for example a k-nearest-neighbors classifier after an LMNN transformation, as demonstrated in the paper.
  • Pair, triplet, and quadruplet learners can be calibrated after training with calibrate_threshold to optimize accuracy or F1-score, making them usable as classifiers on tuple data.
  • The package's test coverage and continuous integration mean each new contribution is checked, so the package can grow without breaking existing algorithms.
  • Because the package is MIT-licensed and distributed through the standard Python package index, it lowers the barrier to using metric learning in production systems and in other open source projects.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The tuple-based API for pairs, triplets, and quadruplets could serve as a template for other weakly supervised estimators, such as rankers or ordinal embedding methods, where supervision also arrives as constraints on relative distances.
  • All ten algorithms optimize a linear transformation, so the package currently leaves out non-linear and deep metric learning; extending the interface to those cases while keeping the same fit/transform pattern would widen its applicability.
  • The scikit-learn compatibility means metric learners can be chained with feature scaling, extraction, and other transformers in a single pipeline, enabling end-to-end optimization of the full data processing chain.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

0 major / 4 minor

Summary. This paper describes metric-learn, an open-source Python package for supervised and weakly-supervised Mahalanobis distance metric learning. The package is part of scikit-learn-contrib and provides a unified API compatible with scikit-learn, enabling cross-validation, model selection, pipelining, and integration with other estimators. The paper presents the package's design, lists the ten implemented algorithms (NCA, LMNN, RCA, LFDA, MLKR, MMC, ITML, SDML, SCML, LSML), describes the supervision types supported, and gives code examples for a supervised pipeline (LMNN + k-NN with grid search) and a weakly supervised pair learner (MMC on LFW pairs). It also reports installation and development details, including a claim of 97% test coverage as of June 2020.

Significance. If the package works as described, it is a valuable contribution to the Python machine learning ecosystem. It fills a gap between Matlab implementations of metric learning algorithms, the R dml package, and Python libraries that focus on either fully supervised or deep metric learning. The scikit-learn-compatible interface makes metric learning easily accessible for practitioners and allows composition with standard tools such as pipelines and grid search. The paper's strengths include the open-source code under an MIT license, availability on PyPI and conda-forge, integration with scikit-learn-contrib, a documented API, and a reported high test coverage. The paper makes no overreaching performance claims and is appropriately positioned relative to existing software. The code snippets are consistent with the described interface and demonstrate practical usage. The absence of independent numerical validation against reference implementations is a typical limitation of software papers and does not undermine the central claim, especially given the public availability of the code and tests.

minor comments (4)
  1. [Section 3] The claim of 97% test coverage is self-reported and the paper does not describe what the tests check or where they can be inspected; adding a sentence pointing to the test suite in the repository and briefly describing the testing strategy (e.g., comparison of learned metrics on synthetic data with known ground truth) would make the 'thoroughly tested' assertion more concrete.
  2. [Section 4] The sentence stating that all metric learners 'should implement two methods: get metric and score pairs' is slightly imprecise, since weakly supervised learners also expose methods such as calibrate_threshold and supervised learners inherit transform from TransformerMixin; clarifying that these are the minimal core methods would avoid confusion.
  3. [Section 4, code snippet] The call to train_test_split in the LFW example does not set a random_state, which makes the example non-deterministic; adding random_state=0 (or an explicit note that the split is random) would improve reproducibility of the illustrative code.
  4. [Section 2] The description of RCA as a supervised learner is qualified by a footnote about chunklets, but the categorization in the main text could be made clearer by mentioning chunklet supervision directly in the list of supervised learners.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity identified; the paper makes implementation claims that are independently checkable against public code and external algorithm references.

full rationale

This is a software description paper, not a derivation or prediction paper. The central claim is that metric-learn implements ten published metric-learning algorithms in a scikit-learn-compatible API (Sections 3 and 4). There is no fitted parameter later renamed as a prediction, no equation whose outcome is fixed by its inputs, and no uniqueness theorem invoked to force a choice. The paper does cite work by its own authors: the metric-learning survey (Bellet et al., 2015), SCML (Shi et al., 2014), and future-work item Liu and Bellet (2019). None of these citations is load-bearing: the survey is background context, SCML is presented as one of several externally published algorithms implemented by the package, and Liu and Bellet is mentioned only as a future direction. The self-reported 97% test coverage and continuous integration in Section 3 are evidence about software engineering practice rather than a circular derivation; the absence of a shown reference comparison is a limitation of the software-paper genre, not a circularity. The code snippets in Section 4 are consistent with the stated API and do not assume the conclusion. Thus the central claim is independently checkable against the public repository and scikit-learn conventions, and no circular step is present.

Assumptions & free parameters 0 free parameters · 1 assumptions · 0 invented entities

The paper makes no parametric or derivational claims, so there are no free parameters. It relies on the standard Mahalanobis distance framework as background, and no new entities are introduced.

assumptions (1)
  • domain assumption All algorithms in metric-learn learn Mahalanobis distances.
    The paper restricts the package to Mahalanobis distances, which is a standard modeling choice in the metric learning literature. This is stated in Section 2 as the framework for all implemented algorithms.

how reviews work

0 comments
Cite this review

Pith. "Pith review of metric-learn: Metric Learning Algorithms in Python." pith.science (2026). https://pith.science/paper/MQELVSAE

@misc{pith2026190804710,
  author       = {Pith},
  title        = {Pith review of: metric-learn: Metric Learning Algorithms in Python},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MQELVSAE}},
  note         = {Machine review of arXiv:1908.04710}
}
read the original abstract

metric-learn is an open source Python package implementing supervised and weakly-supervised distance metric learning algorithms. As part of scikit-learn-contrib, it provides a unified interface compatible with scikit-learn which allows to easily perform cross-validation, model selection, and pipelining with other machine learning estimators. metric-learn is thoroughly tested and available on PyPi under the MIT licence.

Figures

Figures reproduced from arXiv: 1908.04710 by the authors.

Figure 1
Figure 1. Different types of supervision for metric learning illustrated on face image data taken from the Labeled Faces in the Wild data set (Huang et al., 2012). algorithms towards the intended semantics. Finally, metric learning can be used to perform dimensionality reduction. These use-cases highlight the importance of integrating metric learning with the rest of the machine learning pipeline and tools. metric-learn is an… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 17 canonical work pages

  1. [1]

    Bellet, A

    A. Bellet, A. Habrard, and M. Sebban. M etric L earning . M organ & C laypool P ublishers, 2015

    work page 2015

  2. [2]

    J. V. Davis, B. Kulis, P. Jain, S. Sra, and I. S. Dhillon. Information-Theoretic Metric Learning . In ICML, 2007

    work page 2007

  3. [3]

    Goldberger, S

    J. Goldberger, S. Roweis, G. Hinton, and R. Salakhutdinov. Neighbourhood Components Analysis . In NIPS, 2004

    work page 2004

  4. [4]

    G. B. Huang, M. Mattar, H. Lee, and E. Learned-Miller. Learning to Align from Scratch . In NIPS, 2012

    work page 2012

  5. [5]

    E. Y. Liu , Z. Guo , X. Zhang , V. Jojic , and W. Wang . Metric Learning from Relative Comparisons by Minimizing Squared Residual . In ICDM, 2012

    work page 2012

  6. [6]

    Liu and A

    K. Liu and A. Bellet. Escaping the Curse of Dimensionality in Similarity Learning: Efficient Frank-Wolfe Algorithm and Generalization Bounds . Neurocomputing, 333: 0 185--199, 2019

    work page 2019

  7. [7]

    Liu and I

    W. Liu and I. W. Tsang. Large Margin Metric Learning for Multi-Label Prediction . In AAAI, 2015

    work page 2015

  8. [8]

    Paszke, S

    A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala. PyTorch: An Imperative Style, High-Performance Deep Learning Library . In NeurIPS, 2019

    work page 2019

Show all 18 references
  1. [9]

    Pedregosa, G

    F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine Learning in P ython . Journal of Machine Learni...

    2011

  2. [10]

    G.-J. Qi, J. Tang, Z.-J. Zha, T.-S. Chua, and H.-J. Zhang. An Efficient Sparse Metric Learning in High-dimensional Space via L1-penalized Log-determinant Regularization . In ICML, 2009

    2009

  3. [11]

    Shental, T

    N. Shental, T. Hertz, D. Weinshall, and M. Pavel. Adjustment Learning and Relevant Component Analysis . In ECCV, 2002

    2002

  4. [12]

    Y. Shi, A. Bellet, and F. Sha. Sparse Compositional Metric Learning . In AAAI, 2014

    2014

  5. [13]

    Sugiyama

    M. Sugiyama. Dimensionality Reduction of Multimodal Labeled Data by Local Fisher Discriminant Analysis . Journal of Machine Learning Research, 8: 0 1027--1061, 2007

    2007

  6. [14]

    J. L. Suárez, S. García, and F. Herrera. pyDML: A Python Library for Distance Metric Learning . Journal of Machine Learning Research, 96: 0 1--7, 2020

    2020

  7. [15]

    Y. Tang, T. Gao, and N. Xiao. dml: Distance metric learning in R . Journal of Open Source Software, 3 0 (30): 0 1036, 2018

    2018

  8. [16]

    K. Q. Weinberger and L. K. Saul. Distance Metric Learning for Large Margin Nearest Neighbor Classification . Journal of Machine Learning Research, 10: 0 207--244, 2009

    2009

  9. [17]

    K. Q. Weinberger and G. Tesauro. Metric Learning for Kernel Regression . In AISTATS, 2007

    2007

  10. [18]

    E. P. Xing, A. Y. Ng, M. I. Jordan, and S. J. Russell. D istance M etric L earning with A pplication to C lustering with S ide- I nformation. In NIPS, 2002

    2002

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.