REVIEW 4 major objections 5 minor 15 references
An Adjusted Nearest Neighbor Algorithm Maximizing the F-Measure from Imbalanced Data
T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Scaling the distance to positive examples by gamma<1 expands their Voronoi cells, lowers the false-negative rate, and improves the F-measure in k-NN on imbalanced data.
desk verdict A simple, practical kNN variant for imbalanced data with broad experiments, but the theoretical justification is thin and the main baseline comparison may be unfair due to unequal tuning. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The key object is the gamma-scaled distance $d_\gamma(x, x_i)=\gamma d(x,x_i)$ for positive $x_i$ and $d(x,x_i)$ for negatives, with $\gamma\in(0,1)$. Although $d_\gamma$ is not a true metric, it modifies the ranking of neighbors and thereby the Voronoi tessellation and the decision boundary: lower $\gamma$ expands the effective basin of each positive training point. The theoretical engine is a per-point false-negative/false-positive probability written as a product over same-class training points of the probability that the point falls outside a sphere of radius $\epsilon/\gamma$, from which the $\gamma\le 1$ and $\gamma\ge 1$ comparisons follow. In the experimental pipeline, $\gamma$ is selected by cross-validation on the training set and, in the oversampling variant, a second $\gamma$ value is assigned to synthetic positives so the algorithm can down-weight or up-weight them independently.
What would settle it
Run gamma_kNN at k=3 on a public imbalanced dataset and check whether the cross-validated F1 curve over gamma in {0.1, 0.2, ..., 1.0} is flat or decreasing at gamma<1; if gamma<1 never yields higher F1 than gamma=1 under the same search procedure, the central claim is contradicted. Also measure the false-negative rate on a large test set as gamma varies: if it does not fall monotonically for gamma<1 at k=3, the k=1 theory does not extend to the experimental setting.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is that the imbalance problem in nearest-neighbor classification can be addressed by a scalar reweighting of the distance to positives, $d_\gamma(x,x_+) = \gamma d(x,x_+)$, rather than by generating new examples. For each query the algorithm finds the $k$ nearest positives and $k$ nearest negatives under the original distance, scales positive distances by $\gamma$, merges the lists, and takes a majority vote on the $k$ closest; $\gamma<1$ enlarges the effective Voronoi region of each positive. Propositions 1 and 2 show, for $k=1$ and under an independence factorization, that $\gamma\le 1$ decreases the false-negative probability while $\gamma\ge 1$ decreases the false-positive probability; since the minority class is small, the false-positive growth from $\gamma<1$ is comparatively negligible, so the method restricts $\gamma$ to $(0,1)$. Empirically, $\gamma$ tuned by 10-fold cross-validation reaches the best mean F1 among the compared distance-based methods on public data, and applying a second $\gamma$ (typically $>1$) to synthetic positives generated by oversamplers improves the combined pipeline.
Load-bearing premise
The theoretical guarantees are proved only for k=1, while all reported experiments use k=3, and the proof assumes that the false-negative probability factorizes as an independent product over positives with a fixed nearest-neighbor distance epsilon; if that analysis does not carry over to higher k, the theory does not cover the reported results.
Editorial extensions
If this is right
- Tuning a single scalar $\gamma$ via cross-validation raises mean F1 on 19 public datasets from 0.543 (plain k-NN) to 0.607, beating all compared distance-based methods.
- $\gamma k$-NN outperforms each of five oversamplers used alone, and the best results come from applying $\gamma k$-NN after oversampling, because synthetic positives fill empty regions while $\gamma$ rescales the real positives.
- Optimal $\gamma$ decreases as the imbalance ratio grows, so the method automatically compensates for harder imbalance, leveling off at very high imbalance ratios.
- On the private tax-fraud datasets, mean F1 rises from 0.148 for k-NN to 0.278 for $\gamma k$-NN and 0.295 when combined with SMOTE, showing the effect on a real fraud-selection task.
Reading between the lines
- Because the neighbor search is still performed with the original metric and only the merged ranking is scaled, $\gamma k$-NN can be dropped into an approximate-nearest-neighbor index with negligible overhead, and a natural extension is an adaptive $\gamma(x)$ that varies across the feature space.
- The two-gamma treatment of synthetic positives suggests a general principle: any generative or oversampling augmentation can be assigned its own distance scale, allowing the classifier to down-weight low-confidence synthetic points; this idea transfers to GAN-based augmentation without retraining.
- The $k=1$ product formula implies an optimal $\gamma$ roughly tied to local positive density, so a testable extension is to set $\gamma$ per query from the local imbalance ratio and check whether the F1 gains persist.
- The framing of $\gamma$ as a diagonal Mahalanobis matrix restricted to positive comparisons hints that learning a full positive-semidefinite matrix under an F-measure objective could capture direction-dependent imbalance, which the paper itself lists as future work.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes γk-NN, a k-nearest-neighbor variant for imbalanced classification in which the distance from a query to every positive training example is multiplied by a factor γ in (0,1). The authors argue that this expands the Voronoi regions of positive examples, reducing false negatives while controlling false positives, and provide two propositions (Section 4.2) intended to justify the choice γ<1 for the minority class. They evaluate the method on 19 public UCI/KEEL datasets and 11 private DGFiP tax-fraud datasets, comparing against distance-based baselines (k-NN, weighted k-NN, class-weighted k-NN, duplicated-positive k-NN, kRNN, LMNN) and oversampling methods (SMOTE, Borderline-SMOTE, ADASYN, SMOTE+ENN, SMOTE+Tomek), and they report improvements in mean F1, including further gains when γk-NN is combined with oversampling. The central claim is that the distance-rescaling mechanism itself, rather than incidental parameter tuning, is responsible for the reported gains.
Significance. If substantiated, this is a practically attractive contribution: the algorithm is simple, has the same complexity as k-NN, is easy to combine with sampling methods, and is evaluated on a real large-scale tax-fraud application that gives the study ecological validity. The theoretical propositions, although presented only as sketches, correctly capture the intuition that shrinking distances to positives enlarges their effective influence. However, the current evidence has three load-bearing gaps: the theory is proved only for k=1 while all experiments use k=3; the proof relies on an unjustified independence/fixed-ε factorization; and the empirical comparison does not control for the fact that γ is tuned by cross-validation while the main distance-based baselines use fixed influence parameters. The central idea is defensible and the gaps appear fixable, but the manuscript as written does not yet support the strong empirical and theoretical claims made in the abstract and Section 5.
major comments (4)
- [Section 4.2, Eq. (1) and Proposition 1] The proof of Proposition 1 is not a derivation from first principles. Eq. (1) factorizes the false-negative probability as a product over positive training examples, treating each event x' ∈ S_{ε/γ}(z) as independent with the same probability and treating ε (the distance to the nearest neighbor) as fixed. For a random query, ε is itself a random variable that depends on the positions of both classes, and the events for different positives are not independent conditional on ε. The inequality FN_γ(z) ≤ FN(z) for γ≤1 actually follows immediately from monotonicity of the spherical probability in the radius, so the factorization is doing no work. The authors should either supply a rigorous derivation with explicit distributional assumptions or reframe the statement as a heuristic justification rather than a formal proposition.
- [Section 4.2 and Section 5.1] The theoretical analysis is explicitly restricted to k=1, while every reported experiment uses k=3. The statement in Section 5.1 that "the same Analysis can be conducted for other values of k" is not a proof. For k>1 the false-negative event is not "all positives are outside a sphere of radius ε/γ": with k>1, a positive example can be among the k nearest neighbors yet still lose the majority vote, and negatives inside the sphere can also influence the outcome. Thus Propositions 1 and 2, as stated, do not cover the experimental results in Tables 2 and 3. The authors should either extend the analysis to k=3 (or general k) with a precise statement about the majority-vote rule, or report all experiments with k=1 and clearly separate the k=1 theoretical claim from the k=3 empirical claim.
- [Section 5.1, Table 2 and Fig. 4] The main empirical comparison is confounded with hyperparameter tuning. The proposed method tunes γ on a 10-fold CV grid over [0,1], while dupk-NN uses a fixed duplication factor equal to the imbalance ratio and cwk-NN uses a fixed class-proportion weight. Since γ acts essentially as a continuous positive-instance replication/class-weight parameter, the reported mean advantage of γk-NN over dupk-NN (0.607 vs 0.575) may reflect the benefit of tuning a positive-influence parameter rather than the specific Voronoi geometry. The same issue appears in the sampling experiments, where OS+γk-NN tunes two γ values (one for real positives and one for synthetic positives) while the OS baselines use default parameters apart from the sampling ratio. The authors should include controlled baselines with an equivalently tuned positive-influence parameter (e.g., a tuned duplication factor or a tuned positive vote weight) and, if possible, report the performance of γk-NN with γ fixed to a value matched to the baseline's effective influence.
- [Section 5.2, Tables 2 and 3] No statistical significance testing is reported. The headline result is a mean F1 gain of about 3.2 points over the second-best baseline, but with standard deviations in the 0.02–0.05 range and only 19 datasets, a paired test is needed to rule out chance. Moreover, Table 2 shows that γk-NN is the best method on only about 7 of 19 datasets, and the average gain is strongly influenced by the hayes dataset (0.593 vs 0.221 for the next best) and the german dataset. The authors should report per-dataset win/loss counts and a paired significance test (e.g., Wilcoxon signed-rank test) with effect sizes, and they should discuss whether the conclusion is robust when the most influential datasets are removed.
minor comments (5)
- [Section 5.1, text after Table 2] The sentence "The results for k = 1 are comparable as the presented results in this section" is incomplete and no k=1 results are shown anywhere; please either provide the k=1 table or remove the sentence.
- [Section 5.1, experimental setup] The text says the γ parameter is tuned "in the range [0, 1]3 using a step of 0.1"; the superscript 3 appears to be a typo, since γ is a scalar. It should read "in the range [0,1] using a step of 0.1", and the same correction applies to the quoted range for synthetic-positive γ in Section 5.2.
- [Table 3] The table uses comma decimal separators for some entries (e.g., 0,454) but the text and other tables use decimal points; please standardize the decimal notation.
- [Figures 4 and 6] The figure legends/labels use "3NN" where the text refers to γk-NN (e.g., "Comparison of -3NN with 3NN" in Fig. 4 and "-3NN and OS* + -3NN" in Fig. 6). These labels should be clarified to read γk-NN and OS+γk-NN respectively.
- [Algorithm 1] The tie-breaking rule is implicit: for even k, the condition |NN_γ ∩ NN_+| ≥ k/2 means ties are resolved in favor of the positive class. This should be stated explicitly because it biases the algorithm toward recall and is relevant to the F1 comparison.
Circularity Check
No significant circularity: the theoretical statements are monotonicity consequences of the defined distance, and the empirical evaluation is based on independent external benchmarks.
full rationale
The paper's central derivation reduces to Proposition 1: if distances to positives are multiplied by γ < 1, then the probability that a positive is a false negative decreases, because the positive-neighbor region around a query expands. This follows directly from the definition of dγ and the nearest-neighbor rule, but that makes it a valid and self-contained monotonicity theorem, not a circular one: the conclusion is not assumed in the definition of dγ, and the paper does not claim to derive the optimal γ from the theorem — γ is tuned by cross-validation. The empirical contribution is assessed on UCI/KEEL datasets and on DGFiP tax-fraud data against standard baselines, so the central empirical claim does not rely on a fitted parameter being relabeled as a prediction. The self-citations (Bellet et al. 2015; Fréry et al. 2017) are related-work references and are not load-bearing for the proposed method or its evaluation. The known gap that the theory is proved only for k=1 while experiments use k=3 is an unsupported extension, and the uneven hyperparameter tuning of baselines is a comparison fairness concern, but neither is a circularity of the derivation chain. No step of the paper's argument reduces, by construction or by self-citation, to its own inputs.
Assumptions & free parameters
free parameters (3)
- γ (positive distance scaling factor) =
Tuned on validation grid [0.0, 1.0] step 0.1
- γ_synth (scaling for synthetic positives) =
Tuned on validation grid [0.0, 2.0] step 0.1
- Oversampling ratio =
Tuned over {0.1, 0.2, ..., 1.0}
assumptions (3)
- ad hoc to paper The false-negative probability factorizes as a product over independent positive examples with a fixed nearest-neighbor distance ε (Eq. 1).
- domain assumption The k=1 theoretical analysis extends to k=3.
- standard math Conditional on the modified distance dγ, the k-NN decision rule approximates the Bayes classifier asymptotically.
Cite this review
Pith. "Pith review of An Adjusted Nearest Neighbor Algorithm Maximizing the F-Measure from Imbalanced Data." pith.science (2026). https://pith.science/paper/2GK4JU7D
@misc{pith2026190900693,
author = {Pith},
title = {Pith review of: An Adjusted Nearest Neighbor Algorithm Maximizing the F-Measure from Imbalanced Data},
year = {2026},
howpublished = {\url{https://pith.science/paper/2GK4JU7D}},
note = {Machine review of arXiv:1909.00693}
}
read the original abstract
In this paper, we address the challenging problem of learning from imbalanced data using a Nearest-Neighbor (NN) algorithm. In this setting, the minority examples typically belong to the class of interest requiring the optimization of specific criteria, like the F-Measure. Based on simple geometrical ideas, we introduce an algorithm that reweights the distance between a query sample and any positive training example. This leads to a modification of the Voronoi regions and thus of the decision boundaries of the NN algorithm. We provide a theoretical justification about the weighting scheme needed to reduce the False Negative rate while controlling the number of False Positives. We perform an extensive experimental study on many public imbalanced datasets, but also on large scale non public data from the French Ministry of Economy and Finance on a tax fraud detection task, showing that our method is very effective and, interestingly, yields the best performance when combined with state of the art sampling methods.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
Aggarwal, C. C. (2017).Outlier Analysis. Springer International Publishing. Barandela, R., Sánchez, J. S., Garca, V., and Rangel, E. (2003). Strategies for learning in class imbalance problems.Pattern Recognition,
work page 2017
-
[3]
The values correspond to the mean F-measure F1 over 5 runs
13 Table 3: Results for 3−NN on the DGFiP datasets. The values correspond to the mean F-measure F1 over 5 runs. The best result on each dataset is indicated in bold while the second is underlined. datasets 3−NN γk−NN SMOTE SMOTE+γk−NN Dgfip19 2 0,454(0,007) 0,528(0,005) 0,505(0,010) 0,529(0,003) Dgfip9 2 0,173(0,074) 0,396(0,018) 0,340(0,033) 0,419(0,029)...
work page 2002
-
[4]
Fernández, A., Garcia, S., Herrera, F., and Chawla, N. V. (2018). Smote for learning from imbalanced data: Progress and challenges, marking the 15-year anniversary.Journal of Artificial Intelligence Research,
work page 2018
-
[5]
Rijsbergen, C. J. V. (1979).Information Retrieval. Sajjadi, M. S. M., Bachem, O., Lucic, M., Bousquet, O., and Gelly, S. (2018). Assessing generative models via precision and recall. InAdvances in Neural Information Processing Systems
work page 1979
-
[10]
Wilson, D. L. (1972). Asymptotic properties of nearest neighbor rules using edited data.IEEE Transactions on Systems, Man, and Cybernetics,
work page 1972
-
[13]
V., Kulis, B., Jain, P., Sra, S., and Dhillon, I
Davis, J. V., Kulis, B., Jain, P., Sra, S., and Dhillon, I. S. (2007). Information-theoretic metric learning. In ICML. Dudani, S. A. (1976). The distance-weighted k-nearest-neighbor rule.IEEE Transactions on Systems, Man, and Cybernetics,
work page 2007
-
[15]
Zhang, X. and Li, Y. (2013). A positive-biased nearest neighbour algorithm for imbalanced classi- fication. InPacific-Asia Conference on Knowledge Discovery and Data Mining, pages 293–304. Springer. Zhang, X., Li, Y., Kotagiri, R., Wu, L., Tari, Z., and Cheriet, M. (2017). Krnn: k rare-class nearest neighbour classification.Pattern Recognition, 62:33 –
work page 2013
-
[16]
Cover, T. and Hart, P. (1967). Nearest neighbor pattern classification.IEEE Transactions on Information Theory,
work page 1967
Show all 15 references
-
[29]
and Weiss, R
Kontorovich, A. and Weiss, R. (2015). A Bayes consistent 1-NN classifier. InProceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics, volume
2015
-
[30]
Fréry, J., Habrard, A., Sebban, M., Caelen, O., and He-Guelton, L. (2017). Efficient top rank optimization with gradient boosting for supervised anomaly detection. InMachine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2017, Skopje, Macedonia, S...
2017
-
[31]
Steck, H. (2007). Hinge rank loss and the area under the roc curve. In Kok, J. N., Koronacki, J., Mantaras, R. L. d., Matwin, S., Mladenič, D., and Skowron, A., editors,Machine Learning: ECML
2007
-
[36]
A., Khoshgoftaar, T
Bauder, R. A., Khoshgoftaar, T. M., and Hasanin, T. (2018). Data sampling approaches with severely imbalanced big data for medicare fraud detection. In2018 IEEE 30th international conference on tools with artificial intelligence (ICTAI), pages 137–142. IEEE. Bellet, A., Habrard...
2018
-
[38]
and Chawla, S
Liu, W. and Chawla, S. (2011). Class confidence weighted knn algorithms for imbalanced data sets. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pages 345–356. Springer. Luxburg, U. v. and Bousquet, O. (2004). Distance-based classification with lipschitz funct...
2011
-
[61]
Ferri, C., Hernández-Orallo, J., and Modroiu, R. (2009). An experimental comparison of performance measures for classification.Pattern Recognition Letters,
2009
-
[2007]
Tomek, I. (1976). Two modifications of cnn. InIEEE Transactions on Systems Man and Communi- cations. Weinberger, K. Q. and Saul, L. K. (2009). Distance metric learning for large margin nearest neighbor classification.Journal of Machine Learning Research,
1976
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.