Towards Linearization Machine Learning Algorithms

Steve Tueno

REVIEW 5 major objections 5 minor 14 references

Towards Linearization Machine Learning Algorithms

T0 review · 5 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper proposes that supervised learning can be reduced to a linear projection of the target function followed by a ratio-weighted consensus among k nearest neighbors, and reports accuracy gains over standard neural, linear, and…

desk verdict A clean but incremental local-correction rule that is too under-specified to verify and whose reported accuracy gains are unsupported. read the letter →

arxiv 1908.06871 v1 pith:VF4IKY3G submitted 2019-08-14 cs.LG cs.AI

Steve Tueno This is my paper · ORCID

classification cs.LGcs.AI

keywords linearizationmachinelearningmultilinearprojectionknearestneighborssupervisedregressionbinaryclassificationratio-weightedconsensusbaselinecomparison

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces a supervised learning method that first approximates an unknown target function $f$ by a linear (or multilinear) projection $f'(X)=W^tX$, then predicts a new input's output as a consensus among the $k$ nearest neighbors of $f'(x)$ in the projected space. The prediction rule is $y = \frac{1}{k}\sum_{t} y'\, y_{jt}/y'_{jt}$, where each neighbor's true label is rescaled by the ratio of projected values. For binary classification, labels are converted to pseudo-probabilities and the same consensus rule decides the class by a $0.5$ threshold. The author reports that implementations of this approach achieved higher accuracies than standard multilayer perceptron, logistic regression, and random forest classifiers on several benchmark datasets, even without parameter tuning.

What carries the argument

The central object is the multilinear projection $f'(X)=W^tX$ together with the ratio-weighted consensus formula $y = \frac{1}{k}\sum_{t} y'\, y_{jt}/y'_{jt}$. The projection maps each training point $(X_i, y_i)$ to $(X_i, y'_i)$, and the ratio $y'/y'_{jt}$ rescales each neighbor's true label so that the average reflects the local scaling of the projected output space. Nearest-neighbor search is performed on the projected outputs, so the quality of the linear fit determines whether nearby projected points correspond to nearby true outputs. For classification, the recursive Learn procedure iteratively adjusts pseudo-probability targets, making the classification algorithm a wrapper around the same regression machinery.

What would settle it

Evaluate the method on a smooth but strongly nonlinear target such as $y=\sin(10x)$ with dense, noise-free samples; if the ratio-weighted consensus is less accurate than plain k-nearest-neighbors on the original inputs, or if the largest errors occur where projected training values $y'_{jt}$ are near zero, the central assumption fails.

Watch

Extended reading notes

Core claim

The central claim is that an unknown function can be estimated by projecting it onto a linear function in a new space and then performing a ratio-weighted average of neighbor labels. Concretely, after fitting $f'(X)=W^tX$, the algorithm finds the $k$ nearest neighbors of $y' = f'(x)$ among the projected training outputs and computes $y = \frac{1}{k}\sum_{t=j_1}^{j_k} y'\, y_{jt}/y'_{jt}$. The same formula underlies regression and binary classification: for classification, each label $c_i$ is first assigned a random value $p_i$ in $(0,0.5)$ or $(0.5,1)$, a recursive procedure adjusts these pseudo-targets until they are consistent with the consensus rule, and the final prediction is $1$ if the consensus exceeds $0.5$ and $0$ otherwise. The paper's evidence is a set of accuracy comparisons on a dozen standard binary datasets, in which the linearization approach often outperforms standard multilayer perceptron, logistic regression, and random forest classifiers.

Load-bearing premise

The load-bearing premise is that the target function is locally smooth enough that, for nearby points, the ratio of projected values $y'/y'_{jt}$ tracks the ratio of true values; if the linear projection is a poor fit in a region, the ratio weighting can amplify a neighbor's label into a large, wrong correction.

Editorial extensions

If this is right

Regression and binary classification share one prediction rule, so improvements to the consensus estimator apply to both settings.
Training cost is dominated by fitting a linear projection and organizing projected points for nearest-neighbor queries, avoiding iterative deep or ensemble training.
If the reported comparisons hold, the approach can beat multilayer perceptrons, logistic regression, and random forests on datasets where projected neighborhoods are informative, even without tuning.
Multiclass problems can be handled by reducing to binary problems via one-against-all, as the paper notes.
The prediction rule is inspectable: each neighbor contributes a rescaled vote, so a wrong prediction can be traced to specific neighbors and their projected values.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The formula behaves like locally adaptive k-nearest-neighbors: where the linear fit is accurate, the ratios are near $1$ and the estimate reduces to plain neighbor averaging; where the fit is poor, small projected values amplify errors, so the method's practical success hinges on the projection capturing the global trend.
The classification procedure's recursive pseudo-label updates form a fixed-point iteration whose convergence and dependence on initialization are not analyzed; testing different update rules could change accuracy substantially.
A natural extension is to replace the mean with the median or to apply the ratio correction to distances as well as labels, which would soften the influence of neighbors with very small projected values.
Comparing this method against kernel ridge regression or Gaussian processes could isolate whether the gain comes from the linear projection itself or from the locally linear correction that the ratio provides.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper proposes a local correction rule: fit a linear projection, then for a new point find k nearest projected neighbors and predict by averaging y' times the ratio y_jt / y'_jt. That rule is clean and I have not seen it in the cited literature. The self-consistent pseudo-probability training for classification is also unusual. The author states the algorithm in pseudocode and points to a public repo, which is useful. The soft spots are substantial. The pseudocode uses undefined constants ('inc', 'pas') and an undefined threshold ('>> 0'), and it says tune parameters with an optimization algorithm without naming the optimizer, loss, or stopping rule. Classification initializes p_i randomly, so results could change with the seed. A reader cannot reconstruct the exact algorithm, and the accuracy table is therefore not reproducible. The evaluation is too thin to support the abstract's claim of improvements over Spark MLlib. There are no error bars, no statistical tests, no split description, and no random forest baseline despite the abstract promising one. Regression datasets are listed under classification accuracy without explaining how continuous targets were binarized. The natural baselines for a kNN-style method, plain kNN and linear regression, are missing. The citation pattern also omits local linear regression and label propagation, which are the closest relatives. The author is appropriately cautious in the discussion, saying 'without proper parameter tuning' and 'must be further investigated.' So this is a promising direction rather than a demonstrated win. But the paper does not yet provide enough to verify the central claim. It reads like a preliminary technical report. Who is this for? Someone interested in a simple alternative to kNN with a global linear anchor. It is not publishable in its current form. A serious editor would desk-reject, though a referee's report could help the author tighten the algorithm and evaluation. I would not cite it.

Referee Report

5 major / 5 minor

Summary. The paper proposes a supervised learning approach in which an unknown function f(x) is approximated by a linear (or multilinear) projection f'(x), and prediction for a new point x is obtained by taking a weighted consensus among the k nearest neighbors of f'(x) in the projected space, using the formula y = (1/k) * sum(y' * y_jt / y'_jt). Algorithms are sketched for regression and binary classification, and a table of accuracies on five datasets compares the method against Spark MLlib's multilayer perceptrons, logistic regression, and (in the abstract and introduction) random forests. The central claim is that the linearization approach is often more accurate than these baselines, even without parameter tuning.

Significance. If the approach were fully specified and validated, the idea of replacing the original feature-space distance with a one-dimensional projected-space distance and applying a ratio-based consensus rule would be a simple and potentially interesting addition to the non-parametric classification and regression literature. The paper's strengths include a concrete prediction formula and a link to a Scala implementation, both of which could facilitate further investigation. However, the manuscript in its current form does not establish the claimed improvements: the learning algorithm is incompletely specified, the empirical comparison lacks a recoverable protocol, and the key modeling assumptions are neither stated nor tested. The contribution is therefore not yet sufficient for publication.

major comments (5)

[§1.2.3 (Function Learn)] The Learn procedure is not executable as written: the constants 'inc' and 'pas' are never defined, and the predicates '>> 0', '>>', and '<<' are not defined. The recursion also lacks a precise termination condition—'If (p_i) has not changed' does not specify how equality is tested—and no convergence argument is given. Because the final prediction rule for classification depends on the p_i produced by this procedure, the central empirical claim cannot be checked.
[§1.2.2 (Binary Classification)] The binary classification algorithm does not describe how the projection f' (i.e., the coefficients a,b or W) is estimated. The only fitting instruction appears in §1.2.1, where the author writes 'estimate the best parameter values using an optimization algorithm' without specifying the objective function, the optimizer, the initialization, or the stopping rule. Since both the training-time neighbors in Learn and the test-time prediction require f'(x), the algorithm is incomplete as published.
[Table 1] The empirical evaluation in Table 1 is not reproducible. There are no error bars, no number of runs, no description of the train/test split, and no hyperparameter settings for the linearization method or for the Spark MLlib baselines. The random-forest baseline promised in the abstract and introduction is missing from the table. Moreover, the rows 'square root' and 'exp' appear to be regression tasks, yet they are reported as classification accuracies without explaining how continuous targets were converted to classes; the test counts 3507 and 4010 also conflict with the statement that the square-root dataset uses 'a thousand randomly picked points.'
[Eq. (1) (prediction rule)] The prediction formula y = (1/k) * sum_t (y' * y_jt / y'_jt) divides by each neighbor's projected value y'_jt; if any y'_jt is zero (or very small), the estimate is undefined or numerically unstable, and no safeguard is described. More fundamentally, the method relies on the assumption that the ratio y_jt / y'_jt is approximately constant among nearby projected points, but this assumption is not stated formally or validated on the datasets. A poor local fit of the linear projection could amplify errors rather than correct them.
[§1.2.3 (random initialization and convergence)] The Learn procedure initializes each p_i with a random draw from (0,0.5) or (0.5,1) and then iteratively updates p_i using the same training data. Because the test-time prediction in step 3 of §1.2.2 depends on the fitted p_jt, the results in Table 1 are potentially seed-dependent, yet no variance or ensemble results are reported. There is also no proof of convergence to a fixed point, nor any argument that the fixed point, if reached, would generalize to new data.

minor comments (5)

[Throughout] The manuscript uses nonstandard notation such as 'k∈ 1..n' and 'random(]0, 0.5[)'; these should be replaced with conventional mathematical notation (e.g., k ∈ {1,...,n} and uniform draws from intervals).
[Abstract and Discussion] The abstract states that the implementations 'have demonstrated improvements in prediction accuracies,' but the discussion in Section 2 concedes 'without proper parameter tuning' and 'must be further investigated.' The strength of the claim should match the actual evidence presented.
[Eq. (1) and notation] The summation notation in Eq. (1) is unclear: the index t is not defined in the text, and the meaning of the limits j1 and jk is ambiguous. The author should define the indexing used for the k nearest neighbors.
[Datasets] The 'square root' and 'exp' datasets are not standard LIBSVM datasets and no URL or generation procedure is provided beyond a brief footnote; the manuscript should state how these data were created and preprocessed.
[Reproducibility] The paper cites the GitHub repository [9] as the implementation, but it does not specify the exact commit, runtime environment, or data-processing steps needed to reproduce Table 1.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the linearization algorithm is an empirical supervised fitting method whose prediction rule is the method itself, not a result that reduces to its own inputs.

full rationale

The paper's central claim is that a particular prediction rule, y = (1/k) * sum(y' * y_jt / y'_jt), built on a fitted linear projection f'(x) and k nearest neighbors in projected space, gives competitive accuracy. This rule is presented as the algorithm's definition rather than as a derived consequence of an external principle, so there is no step where the conclusion is loaded into the premises. The regression procedure fits parameters (a,b) or W by an optimization algorithm and then uses those fitted parameters to form predictions; this is ordinary supervised learning, not a fitted input renamed as a prediction. In the binary classification variant, the p_i pseudo-probabilities are initialized from training labels and refined by the Learn procedure, and the final prediction uses those fitted values; again, this is a standard empirical fitting loop, not circular reasoning. The paper does not invoke any uniqueness theorem, and its only self-referential item, reference [9], is merely the GitHub implementation of the proposed approach, not a load-bearing citation that justifies the method's validity. Concerns that the algorithm is underspecified (undefined constants inc and pas, the threshold predicate '>> 0', unspecified optimizer, random initialization, and missing evaluation protocol) are reproducibility and correctness issues, not circularity. The empirical comparisons in Table 1 are external benchmarks against Spark MLlib methods; even if the comparison is incomplete or ambiguous, nothing in the paper reduces the claimed accuracy gain to the method's own definition by construction. Accordingly, the appropriate circularity finding is a clean non-finding with score 0.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The central claim depends on several unspecified hyperparameters (k, the projection coefficients, thresholds and increments in Learn) and on the untested assumption that a linear projection preserves the local ratios used in the prediction rule. For classification, each training point's pseudo-probability is an additional fitted parameter, so the model's capacity is high and its behavior is not characterized.

free parameters (5)

k (number of nearest neighbors)
A hyperparameter of the algorithm; no tuning procedure or final values are reported.
Projection coefficients W (or a, b)
Coefficients of the linear function f' are said to be 'fine tuned using optimization algorithms', but no fitting method or resulting values are specified.
Increment parameter 'inc' or 'pas'
Used in the Learn function to adjust pseudo-probabilities; the value is unspecified.
Threshold for '>>' and '<<'
The Learn function uses magnitude comparisons such as '|qi - pi| >> 0' without defining the threshold.
Per-data-point pseudo-probabilities p_i = Trained per dataset; values not reported
For classification, each training point receives a pseudo-probability that is randomly initialized and then updated to satisfy a self-consistency condition; these are effectively n additional fitted parameters.

assumptions (3)

domain assumption The unknown function f can be approximated by a linear or multilinear function f' in some projected space E'.
The method assumes this projection makes the kNN consensus accurate; this is stated in Section 1.1 but never justified or tested.
ad hoc to paper The ratio y_jt / y'_jt is consistent among nearby points, so weighting by y'/y'_jt is valid.
This is implicit in the prediction formula (Eq. 1); no error analysis is given.
ad hoc to paper The iterative Learn procedure converges to a fixed point and the fixed point generalizes to new data.
The recursion in Section 1.2.3 is not proved to terminate or converge, and its generalization properties are not analyzed.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Towards Linearization Machine Learning Algorithms." pith.science (2026). https://pith.science/paper/VF4IKY3G

@misc{pith2026190806871,
  author       = {Pith},
  title        = {Pith review of: Towards Linearization Machine Learning Algorithms},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VF4IKY3G}},
  note         = {Machine review of arXiv:1908.06871}
}

read the original abstract

This paper is about a machine learning approach based on the multilinear projection of an unknown function (or probability distribution) to be estimated towards a linear (or multilinear) dimensional space E'. The proposal transforms the problem of predicting the target of an observation x into a problem of determining a consensus among the k nearest neighbors of x's image within the dimensional space E'. The algorithms that concretize it allow both regression and binary classification. Implementations carried out using Scala/Spark and assessed on a dozen LIBSVM datasets have demonstrated improvements in prediction accuracies in comparison with other prediction algorithms implemented within Spark MLLib such as multilayer perceptrons, logistic regression classifiers and random forests.

Figures

Figures reproduced from arXiv: 1908.06871 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

14 extracted references · 12 canonical work pages

[1]

Breiman, Classiﬁcation and regression trees, Routledge, 2017

L. Breiman, Classiﬁcation and regression trees, Routledge, 2017

2017
[2]

J. R. Quinlan, et al., Learning with continuous classes, in: 5th Australian joint conference on artiﬁcial intelligence, V ol. 92, World Scientiﬁc, 1992, pp. 343–348

work page 1992
[3]

G. A. Seber, A. J. Lee, Linear regression analysis, V ol. 329, John Wiley & Sons, 2012

work page 2012
[4]

D. G. Kleinbaum, K. Dietz, M. Gail, M. Klein, M. Klein, Logistic regression, Springer, 2002

work page 2002
[5]

Breiman, Random forests, Machine learning 45 (1) (2001) 5–32

L. Breiman, Random forests, Machine learning 45 (1) (2001) 5–32. 4

work page 2001
[6]

Rokach, O

L. Rokach, O. Z. Maimon, Data mining with decision trees: theory and applica- tions, V ol. 69, World scientiﬁc, 2008

work page 2008
[7]

S. K. Pal, S. Mitra, Multilayer perceptron, fuzzy sets, and classiﬁcation, IEEE Transactions on neural networks 3 (5) (1992) 683–697

work page 1992
[8]

C. M. Bishop, Pattern recognition and machine learning, springer, 2006

2006

Show all 14 references

[9]

URL https://github.com/stuenofotso/LinearizationML

Scala implementation of the linearization machine learning approach. URL https://github.com/stuenofotso/LinearizationML
[10]

Odersky, L

M. Odersky, L. Spoon, B. Venners, Programming in scala, Artima Inc, 2008

2008
[11]

Pentreath, Machine learning with spark, Packt Publishing Ltd, 2015

N. Pentreath, Machine learning with spark, Packt Publishing Ltd, 2015

2015
[12]

Zeng, H.-B

Z.-Q. Zeng, H.-B. Yu, H.-R. Xu, Y .-Q. Xie, J. Gao, Fast training support vector machines using parallel sequential minimal optimization, in: 2008 3rd international conference on intelligent system and knowledge engineering, V ol. 1, IEEE, 2008, pp. 997–1001

2008
[13]

URL https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/ binary.html

LIBSVM Data: Classiﬁcation (Binary Class) (2019). URL https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/ binary.html

2019
[14]

X. Meng, J. Bradley, B. Yavuz, E. Sparks, S. Venkataraman, D. Liu, J. Freeman, D. Tsai, M. Amde, S. Owen, et al., Mllib: Machine learning in apache spark, The Journal of Machine Learning Research 17 (1) (2016) 1235–1241. 5

2016

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Breiman, Classiﬁcation and regression trees, Routledge, 2017

L. Breiman, Classiﬁcation and regression trees, Routledge, 2017

2017

[2] [2]

J. R. Quinlan, et al., Learning with continuous classes, in: 5th Australian joint conference on artiﬁcial intelligence, V ol. 92, World Scientiﬁc, 1992, pp. 343–348

work page 1992

[3] [3]

G. A. Seber, A. J. Lee, Linear regression analysis, V ol. 329, John Wiley & Sons, 2012

work page 2012

[4] [4]

D. G. Kleinbaum, K. Dietz, M. Gail, M. Klein, M. Klein, Logistic regression, Springer, 2002

work page 2002

[5] [5]

Breiman, Random forests, Machine learning 45 (1) (2001) 5–32

L. Breiman, Random forests, Machine learning 45 (1) (2001) 5–32. 4

work page 2001

[6] [6]

Rokach, O

L. Rokach, O. Z. Maimon, Data mining with decision trees: theory and applica- tions, V ol. 69, World scientiﬁc, 2008

work page 2008

[7] [7]

S. K. Pal, S. Mitra, Multilayer perceptron, fuzzy sets, and classiﬁcation, IEEE Transactions on neural networks 3 (5) (1992) 683–697

work page 1992

[8] [8]

C. M. Bishop, Pattern recognition and machine learning, springer, 2006

2006