pith. sign in

arxiv: 1907.11277 · v1 · pith:U2B7XB7Gnew · submitted 2019-07-25 · 💻 cs.LG · stat.ML

Towards meta-learning for multi-target regression problems

Gabriel Jonas Aguiar , Everton Jos\'e Santana , Saulo Martiello Mastelini , Rafael Gomes Mantovani , Sylvio Barbon Jr This is my paper

Pith reviewed 2026-05-24 16:02 UTC · model grok-4.3

classification 💻 cs.LG stat.ML
keywords meta-learningmulti-target regressionmethod recommendationsynthetic datasetsrandom forestbalanced accuracyinter-target correlationmeta-features
0
0 comments X

The pith

A meta-learning system recommends the best multi-target regression method using meta-features from synthetic data with over 70 percent balanced accuracy.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops a meta-learning system to automatically select the most suitable multi-target regression method for any given problem. Different methods exist that handle inter-target correlations in distinct ways, yet none dominates every dataset. To train the system the authors generate 648 synthetic datasets that systematically vary those correlations and other properties, then extract 58 meta-features from each. Four algorithms are tested as meta-learners; random forest reaches balanced accuracy above 70 percent and beats the baselines. A sympathetic reader would care because the approach replaces manual method trials with an automatic recommendation step for new multi-target tasks.

Core claim

Induced meta-models trained on a meta-dataset of 648 synthetic multi-target regression problems described by 58 meta-features can recommend the best base-level method with balanced accuracy superior to 70 percent when random forest serves as the meta-learner, statistically outperforming the meta-learning baselines.

What carries the argument

Meta-learning system that maps 58 meta-features (statistical, correlation, linear landmarking, distribution and smoothness) extracted from synthetic datasets to the best multi-target regression algorithm among four meta-labels.

If this is right

  • Random forest meta-models outperform the tested meta-learning baselines on the synthetic meta-dataset.
  • Meta-features based on inter-target correlations, linear landmarking and data distribution enable reliable method recommendation.
  • Synthetic datasets engineered to vary inter-target characteristics suffice to produce meta-models that generalize across base-level problems.
  • The system works for four different meta-labels corresponding to distinct multi-target regression methods.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the meta-features transfer to real data, the same pipeline could be applied to new multi-target problems without retraining from scratch.
  • The approach might lower the cost of method selection on large-scale problems where running every candidate regressor is expensive.
  • Adding meta-features that directly measure computational runtime or memory use could extend the system beyond accuracy alone.

Load-bearing premise

The chosen synthetic datasets and 58 meta-features capture the inter-target correlation structures that actually determine which method performs best on real data.

What would settle it

Evaluating the trained meta-model on a large collection of real multi-target regression datasets and measuring recommendation accuracy significantly below 70 percent would falsify the central claim.

Figures

Figures reproduced from arXiv: 1907.11277 by Everton Jos\'e Santana, Gabriel Jonas Aguiar, Rafael Gomes Mantovani, Saulo Martiello Mastelini, Sylvio Barbon Jr.

Figure 1
Figure 1. Figure 1: Overview of the procedure to select a Multi-target Regression method [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: In this figure, each line represents a meta-model and [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Comparison of the aRRMSE values obtained by meta-models when [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Average relative importance of the meta-features obtained from RF importance. The names of the meta-features in the x-axis follow the acronyms [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
read the original abstract

Several multi-target regression methods were devel-oped in the last years aiming at improving predictive performanceby exploring inter-target correlation within the problem. However, none of these methods outperforms the others for all problems. This motivates the development of automatic approachesto recommend the most suitable multi-target regression method. In this paper, we propose a meta-learning system to recommend the best predictive method for a given multi-target regression problem. We performed experiments with a meta-dataset generated by a total of 648 synthetic datasets. These datasets were created to explore distinct inter-targets characteristics toward recommending the most promising method. In experiments, we evaluated four different algorithms with different biases as meta-learners. Our meta-dataset is composed of 58 meta-features, based on: statistical information, correlation characteristics, linear landmarking, from the distribution and smoothness of the data, and has four different meta-labels. Results showed that induced meta-models were able to recommend the best methodfor different base level datasets with a balanced accuracy superior to 70% using a Random Forest meta-model, which statistically outperformed the meta-learning baselines.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper proposes a meta-learning system to recommend the best multi-target regression (MTR) method for a given problem. It generates 648 synthetic datasets varying inter-target characteristics, extracts 58 meta-features (statistical, correlation, landmarking, distribution/smoothness), and trains four meta-learners; a Random Forest achieves >70% balanced accuracy in predicting the best base MTR method and statistically outperforms the meta-learning baselines.

Significance. If the meta-models generalize beyond the synthetic generator, the approach could automate selection among MTR methods that exploit inter-target correlations. The controlled synthetic design to isolate correlation structures is a methodological strength for internal validity, but the absence of real-data transfer testing limits assessed significance for practical MTR problems.

major comments (2)
  1. [Abstract] Abstract: the headline claim that the RF meta-model recommends the best method 'for different base level datasets' with >70% balanced accuracy is evaluated only on the 648 synthetic datasets; no evaluation on a hold-out set of real MTR datasets is reported to test transfer when inter-target structures differ from those spanned by the synthetic generator.
  2. [Experiments] Experiments section (as described in abstract): the abstract reports concrete performance numbers and a statistical comparison yet provides no detail on the synthetic data generation process, the precise definition and computation of the 58 meta-features, or the exact statistical test, leaving the central performance claim only partially reproducible.
minor comments (1)
  1. [Abstract] The abstract could explicitly state the scope (synthetic-only evaluation) to avoid implying direct applicability to real MTR problems.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments. We address each major comment point-by-point below, providing the strongest honest defense of the manuscript while agreeing where revisions are warranted to improve clarity and reproducibility.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the headline claim that the RF meta-model recommends the best method 'for different base level datasets' with >70% balanced accuracy is evaluated only on the 648 synthetic datasets; no evaluation on a hold-out set of real MTR datasets is reported to test transfer when inter-target structures differ from those spanned by the synthetic generator.

    Authors: The manuscript's core contribution is a controlled study using synthetic data to isolate the effects of inter-target correlation structures on MTR method performance, enabling causal insights that real datasets would confound. The abstract accurately reports results on these 648 datasets. We agree that transfer performance on real data remains untested and constitutes a limitation for practical claims. In revision we will (i) rephrase the abstract to explicitly state the synthetic scope and (ii) add a limitations subsection discussing the synthetic-to-real gap and suggesting future validation on real MTR benchmarks. revision: partial

  2. Referee: [Experiments] Experiments section (as described in abstract): the abstract reports concrete performance numbers and a statistical comparison yet provides no detail on the synthetic data generation process, the precise definition and computation of the 58 meta-features, or the exact statistical test, leaving the central performance claim only partially reproducible.

    Authors: The full manuscript contains descriptions of the generator, meta-feature categories, and statistical comparisons, yet we accept that the level of detail is insufficient for exact reproduction. In the revised manuscript we will expand the relevant sections with: explicit parameter ranges and pseudocode for the synthetic generator, precise definitions/formulas (or code references) for all 58 meta-features, and the exact statistical procedure (test name, multiple-comparison correction, and significance threshold) used for the reported comparisons. revision: yes

Circularity Check

0 steps flagged

No circularity: meta-learner trained on independent meta-features to predict externally evaluated best method

full rationale

The paper generates 648 synthetic datasets, computes 58 meta-features (statistical, correlation, landmarking, distribution/smoothness) on each, and assigns meta-labels by running multiple base MTR methods and recording which one wins on that dataset. Four meta-learners are trained to predict the meta-label from the meta-features, with the reported >70% balanced accuracy obtained via cross-validation on these synthetics. No equations define a quantity in terms of itself, no fitted parameter is relabeled as a prediction, and no self-citation chain is invoked to justify uniqueness or an ansatz. The central result is an empirical performance measurement on held-out synthetic instances whose labels are computed independently of the meta-model; it does not reduce to the inputs by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The central empirical claim rests on the representativeness of the synthetic data and the informativeness of the 58 meta-features; no explicit free parameters, axioms, or invented entities are stated in the abstract.

pith-pipeline@v0.9.0 · 5737 in / 1135 out tokens · 22874 ms · 2026-05-24T16:02:35.847925+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

34 extracted references · 34 canonical work pages

  1. [1]

    Multi-target regression via random linear target combinations,

    G. Tsoumakas, E. Spyromitros-Xioufis, A. Vrekou, and I. Vlahavas, “Multi-target regression via random linear target combinations,” in Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 2014, pp. 225–240

    work page 2014

  2. [2]

    Semi-supervised learning for multi-target regression,

    J. Levati ´c, M. Ceci, D. Kocev, and S. D ˇzeroski, “Semi-supervised learning for multi-target regression,” in International Workshop on New Frontiers in Mining Complex Patterns . Springer, 2014, pp. 3–18. 2.5 5.0 7.5 10.0 12.5 min.linearity.regr.L1min.linearity.regr.L3 sd.correlation.C1 min.linearity.regr.L2min.correlation.C2sd.correlation.C2 min.smoothn...

    work page 2014

  3. [3]

    Predicting poultry meat character- istics using an enhanced multi-target regression method,

    E. J. Santana, B. C. Geronimo, S. M. Mastelini, R. H. Carvalho, D. F. Barbin, E. I. Ida, and S. Barbon, “Predicting poultry meat character- istics using an enhanced multi-target regression method,” Biosystems Engineering, vol. 171, pp. 193 – 204, 2018

    work page 2018

  4. [4]

    Multi-output tree chaining: An interpretative modelling and lightweight multi-target approach,

    S. M. Mastelini, V . G. T. da Costa, E. J. Santana, F. K. Nakano, R. C. Guido, R. Cerri, and S. Barbon, “Multi-output tree chaining: An interpretative modelling and lightweight multi-target approach,” Journal of Signal Processing Systems , pp. 1–25, 2018

    work page 2018

  5. [5]

    Stock portfolio prediction by multi-target decision support,

    E. J. Santana, J. A. P. R. d. Silva, S. M. Mastelini, and S. Barbon Jr., “Stock portfolio prediction by multi-target decision support,” iSys- Revista Brasileira de Sistemas de Informac ¸˜ao, vol. 12, no. 1, 2019

    work page 2019

  6. [6]

    On the predictive power of meta-features in openml,

    B. Bilalli, A. Abell ´o, and T. Aluja-Banet, “On the predictive power of meta-features in openml,” International Journal of Applied Mathematics and Computer Science , vol. 27, no. 4, pp. 697–712, 2017

    work page 2017

  7. [7]

    Metalearning and recommender systems: A literature review and empirical study on the algorithm selection problem for collaborative filtering,

    T. Cunha, C. Soares, and A. C. de Carvalho, “Metalearning and recommender systems: A literature review and empirical study on the algorithm selection problem for collaborative filtering,” Information Sciences, vol. 423, pp. 128–144, 2018

    work page 2018

  8. [8]

    Brazdil, C

    P. Brazdil, C. Giraud-Carrier, C. Soares, and R. Vilalta, Metalearning: Applications to Data Mining , 2nd ed. Springer Verlag, 2009

    work page 2009

  9. [9]

    A meta-learning approach to automatic kernel selection for support vector machines,

    S. Ali and K. A. Smith-Miles, “A meta-learning approach to automatic kernel selection for support vector machines,” Neurocomputing, vol. 70, no. 1-3, pp. 173–186, 2006

    work page 2006

  10. [10]

    Metastream: A meta-learning based method for periodic algo- rithm selection in time-changing data,

    A. L. D. Rossi, A. C. P. de Leon Ferreira, C. Soares, B. F. De Souza et al. , “Metastream: A meta-learning based method for periodic algo- rithm selection in time-changing data,” Neurocomputing, vol. 127, pp. 52–64, 2014

    work page 2014

  11. [11]

    Meta-learning for evolutionary parameter optimization of classifiers,

    M. Reif, F. Shafait, and A. Dengel, “Meta-learning for evolutionary parameter optimization of classifiers,” Machine learning, vol. 87, no. 3, pp. 357–380, 2012

    work page 2012

  12. [12]

    Multi-target regression via input space expansion: treating targets as inputs,

    E. Spyromitros-Xioufis, G. Tsoumakas, W. Groves, and I. Vlahavas, “Multi-target regression via input space expansion: treating targets as inputs,” Machine Learning, vol. 104, no. 1, pp. 55–98, 2016

    work page 2016

  13. [13]

    Benchmarking multi-target regression methods,

    S. M. Mastelini, E. J. Santana, V . G. T. da Costa, and S. Barbon, “Benchmarking multi-target regression methods,” in 2018 7th Brazilian Conference on Intelligent Systems (BRACIS). IEEE, 2018, pp. 396–401

    work page 2018

  14. [14]

    The lack of a priori distinctions between learning algorithms,

    D. H. Wolpert, “The lack of a priori distinctions between learning algorithms,” Neural computation, vol. 8, no. 7, pp. 1341–1390, 1996

    work page 1996

  15. [15]

    The algorithm selection problem,

    J. R. Rice, “The algorithm selection problem,” Advances in Computers , vol. 15, pp. 65–118, 1976

    work page 1976

  16. [16]

    Clustering algorithm selection by meta-learning systems: A new distance-based problem characterization and ranking combination methods,

    D. G. Ferrari and L. N. De Castro, “Clustering algorithm selection by meta-learning systems: A new distance-based problem characterization and ranking combination methods,” Information Sciences, vol. 301, pp. 181–194, 2015

    work page 2015

  17. [17]

    A meta-learning approach for recommendation of image segmentation algorithms,

    G. F. Campos, S. Barbon, and R. G. Mantovani, “A meta-learning approach for recommendation of image segmentation algorithms,” in 2016 29th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI). IEEE, 2016, pp. 370–377

    work page 2016

  18. [18]

    A meta-learning recommender system for hyperparameter tuning: Predicting when tuning improves svm classifiers,

    R. G. Mantovani, A. L. Rossi, E. Alcobac ¸a, J. Vanschoren, and A. C. de Carvalho, “A meta-learning recommender system for hyperparameter tuning: Predicting when tuning improves svm classifiers,” Information Sciences, vol. 501, pp. 193 – 221, 2019

    work page 2019

  19. [19]

    A survey on multi-output regression,

    H. Borchani, G. Varando, C. Bielza, and P. Larra ˜naga, “A survey on multi-output regression,” Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery , vol. 5, no. 5, pp. 216–233, 2015

    work page 2015

  20. [20]

    Multi-target support vector regression via correlation regressor chains,

    G. Melki, A. Cano, V . Kecman, and S. Ventura, “Multi-target support vector regression via correlation regressor chains,” Information Sciences, vol. 415, pp. 53–69, 2017

    work page 2017

  21. [21]

    DSTARS: A multi-target deep structure for tracking asynchronous regressor stack,

    S. M. Mastelini, E. J. Santana, R. Cerri, and S. Barbon, “DSTARS: A multi-target deep structure for tracking asynchronous regressor stack,” in 2017 Brazilian Conference on Intelligent Systems (BRACIS) . IEEE, oct 2017

    work page 2017

  22. [22]

    An evolutionary algorithm for optimizing the target ordering in ensemble of regressor chains,

    J. M. Moyano, E. L. Gibaja, and S. Ventura, “An evolutionary algorithm for optimizing the target ordering in ensemble of regressor chains,” in 2017 IEEE Congress on Evolutionary Computation (CEC) . IEEE, jun 2017

    work page 2017

  23. [23]

    Multi-label classification: An overview,

    G. Tsoumakas and I. Katakis, “Multi-label classification: An overview,” International Journal of Data Warehousing and Mining (IJDWM), vol. 3, no. 3, pp. 1–13, 2007

    work page 2007

  24. [24]

    Towards a method for automatically selecting and configuring multi-label classification algo- rithms,

    A. G. de S ´a, G. L. Pappa, and A. A. Freitas, “Towards a method for automatically selecting and configuring multi-label classification algo- rithms,” in Proceedings of the Genetic and Evolutionary Computation Conference Companion. ACM, 2017, pp. 1125–1132

    work page 2017

  25. [25]

    Meta-learning for selecting a multi-label classification algorithm,

    L. Chekina, L. Rokach, and B. Shapira, “Meta-learning for selecting a multi-label classification algorithm,” in 2011 IEEE 11th International Conference on Data Mining Workshops . IEEE, 2011, pp. 220–227

    work page 2011

  26. [26]

    Data complety meta-features for regression problems,

    A. C. Lorena, A. I. Maciel, P. B. de Miranda, I. G. Costa, and R. B. Prudˆencio, “Data complety meta-features for regression problems,” Machine Learning, vol. 107, no. 1, pp. 209–246, 2018

    work page 2018

  27. [27]

    Towards reproducible empirical research in meta-learning,

    A. Rivolli, L. P. F. Garcia, C. Soares, J. Vanschoren, and A. C. P. L. F. de Carvalho, “Towards reproducible empirical research in meta-learning,” CoRR, vol. abs/1808.10406, 2018. [Online]. Available: http://arxiv.org/abs/1808.10406

    work page arXiv 2018

  28. [28]

    Deep Regressor Stacking for Air Ticket Prices Prediction,

    E. J. Santana, S. M. Mastelini, and S. Barbon Jr., “Deep Regressor Stacking for Air Ticket Prices Prediction,” in XIII Brazilian Symposium on Information Systems: Information Systems for Participatory Digital Governance. Brazilian Computer Society (SBC), 2017, pp. 25–31

    work page 2017

  29. [29]

    S. J. Russell and P. Norvig, Artificial intelligence: a modern approach. Malaysia; Pearson Education Limited,, 2016

    work page 2016

  30. [30]

    Random forests,

    L. Breiman, “Random forests,” Mach. Learn., vol. 45, no. 1, pp. 5–32, Oct. 2001

    work page 2001

  31. [31]

    Vapnik, The Nature of Statistical Learning Theory

    V . Vapnik, The Nature of Statistical Learning Theory . New York: Springer-Verlag, 1995

    work page 1995

  32. [32]

    Xgboost: A scalable tree boosting system,

    T. Chen and C. Guestrin, “Xgboost: A scalable tree boosting system,” in Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . ACM, 2016, pp. 785–794

    work page 2016

  33. [33]

    Statistical comparisons of classifiers over multiple data sets,

    J. Dem ˇsar, “Statistical comparisons of classifiers over multiple data sets,” The Journal of Machine Learning Research , vol. 7, pp. 1–30, 2006

    work page 2006

  34. [34]

    Tree-based methods for online multi-target regression,

    A. Osojnik, P. Panov, and S. D ˇzeroski, “Tree-based methods for online multi-target regression,” Journal of Intelligent Information Systems , vol. 50, no. 2, pp. 315–339, 2018

    work page 2018