Pith. sign in

REVIEW 2 major objections 42 references

Reviewed by Pith at T0; open to challenge.

T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →

T0 review · grok-4.3

An online platform recommends complete machine learning pipelines to non-experts by reasoning over encoded expert knowledge.

2026-06-27 20:12 UTC pith:NLWEP7DZ

load-bearing objection This is a high-level design sketch for an FOL-driven ML pipeline recommender with no rules, examples, or implementation shown. the 2 major comments →

arxiv 2606.08212 v1 pith:NLWEP7DZ submitted 2026-06-06 cs.LG

Public Machine Learning Solver Framework for Novices in the Machine Learning Domain

classification cs.LG
keywords machine learning for novicesAutoMLexpert systemsdecision support systemspipeline recommendationfirst-order logictransfer learningcrowdsourcing
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes a public platform that helps novices solve machine learning problems by suggesting full pipelines instead of single algorithms. It merges expert selection criteria with automatic extraction of data features such as class imbalance and missing values, then applies first-order logic to rank recommendations drawn from a knowledge base. Transfer learning supplements the process, and a crowdsourcing component allows experts to keep the rules current. The system is presented as the first free, accessible online framework that makes expert guidance structured and transparent for users without deep ML experience.

Core claim

The authors describe a semi-automated platform that combines expert cheat sheets and decision-support criteria to recommend complete machine learning solution pipelines. The platform automatically extracts dataset characteristics, incorporates transfer learning, and employs first-order logic to reason over a knowledge base of algorithms and selection rules, producing ranked recommendations through a user-friendly interface that can be extended incrementally and updated via expert crowdsourcing.

What carries the argument

First-order logic reasoning over an expert knowledge base of algorithms and selection criteria, paired with automatic data feature extraction and transfer learning to generate ranked pipeline recommendations.

Load-bearing premise

Expert knowledge about suitable algorithms and pipelines can be captured in first-order logic rules that, together with extracted data features, produce recommendations appropriate for non-experts.

What would settle it

A controlled comparison in which the platform and existing AutoML tools each generate pipelines for the same set of real user datasets, followed by expert evaluation of recommendation suitability and downstream model performance.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Non-experts receive ranked lists of complete pipelines rather than isolated algorithm suggestions.
  • Recommendations incorporate multiple criteria such as accuracy, transparency, and data requirements at once.
  • The knowledge base can be extended with new algorithms and domain rules without rebuilding the system.
  • Continuous updates occur through an integrated crowdsourcing platform for machine learning experts.
  • The platform remains publicly accessible and free while maintaining transparency through explicit logical rules.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Domain experts from other fields could apply machine learning more consistently if the recommendations prove reliable on their data types.
  • The explicit logical encoding might allow users to inspect and understand the reasons behind each pipeline suggestion.
  • Similar rule-based guidance structures could be developed for other technical domains that currently require specialized training.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 0 minor

Summary. The manuscript proposes a Public Machine Learning Solver Framework that integrates expert cheat sheets and decision-support criteria into a semi-automated online platform. It uses first-order logic reasoning over a crowdsourced knowledge base, automatic extraction of dataset characteristics (class imbalance, missing values, etc.), and transfer learning to recommend complete ranked ML pipelines to non-experts, claiming to be the first free public system that systematically operationalizes expert knowledge in this manner.

Significance. If the described architecture were implemented with concrete, validated rules and shown to produce suitable recommendations superior to existing AutoML or cheat-sheet approaches, the work could offer a transparent, incrementally updatable alternative for novices. The current manuscript, however, supplies only an architectural outline with no rule set, no feature-extractor implementation, and no reasoning trace, so the claimed significance remains unrealized.

major comments (2)
  1. [Abstract / Platform Architecture] Abstract and platform-architecture section: the central claim that the system 'operationalizes expert knowledge' via first-order logic rules combined with automatic feature extraction and transfer learning is unsupported, as the manuscript contains neither the rule set, pseudocode for the extractors, nor any worked example showing that extracted features plus logic produce ranked pipelines on even one sample dataset.
  2. [Abstract] The assertion that the platform 'suggests a complete pipeline tailored to the user's problem' and is 'superior to existing approaches' rests entirely on the untested assumption that the FOL reasoner will yield suitable outputs; without any implementation or comparison data this assertion cannot be evaluated.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed and constructive comments. The manuscript presents a conceptual proposal for a public ML solver framework rather than a fully implemented and evaluated system. We address each major comment below and will revise the manuscript to clarify the scope and add illustrative details.

read point-by-point responses
  1. Referee: [Abstract / Platform Architecture] Abstract and platform-architecture section: the central claim that the system 'operationalizes expert knowledge' via first-order logic rules combined with automatic feature extraction and transfer learning is unsupported, as the manuscript contains neither the rule set, pseudocode for the extractors, nor any worked example showing that extracted features plus logic produce ranked pipelines on even one sample dataset.

    Authors: We agree that the manuscript supplies only a high-level architectural outline without concrete rule sets, extractor pseudocode, or worked examples. This reflects the paper's focus on the overall design and crowdsourced extensibility rather than a complete implementation. In revision we will add pseudocode for the data-characteristic extractors and a concrete worked example (including extracted features, sample FOL rules, and resulting ranked pipeline) on one public dataset to illustrate the reasoning process. revision: yes

  2. Referee: [Abstract] The assertion that the platform 'suggests a complete pipeline tailored to the user's problem' and is 'superior to existing approaches' rests entirely on the untested assumption that the FOL reasoner will yield suitable outputs; without any implementation or comparison data this assertion cannot be evaluated.

    Authors: The current wording in the abstract presents the pipeline suggestion and potential superiority as design objectives of the proposed framework. We acknowledge that these remain untested without implementation or benchmarks. We will revise the abstract and architecture section to state these as intended outcomes of the architecture, to be validated once the knowledge base and reasoner are populated, rather than as demonstrated results. revision: yes

Circularity Check

0 steps flagged

High-level system design proposal with no derivations or predictions

full rationale

The manuscript is an architectural proposal for an FOL-based recommender platform. It outlines categories of existing systems, describes desired features (feature extraction, transfer learning, crowdsourcing KB updates), and asserts novelty, but supplies no equations, no fitted parameters, no predictions, and no derivation chain. The central claim is a 'first such framework' assertion, not a result derived from inputs. No self-citations, ansatzes, or reductions to fitted data appear. This is a standard non-circular design sketch.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

The proposal rests on the domain assumption that expert ML selection knowledge can be captured in first-order logic and that automatic feature extraction plus crowdsourcing will maintain relevance; no free parameters or invented entities with independent evidence are introduced beyond the platform itself.

axioms (1)
  • domain assumption Expert knowledge for algorithm and pipeline selection can be represented as first-order logic rules over data characteristics and criteria such as accuracy and transparency.
    Invoked when the platform reasons over its knowledge base to produce recommendations.
invented entities (1)
  • Public ML Solver Framework platform no independent evidence
    purpose: To deliver semi-automated, ranked pipeline recommendations to non-experts by integrating expert criteria, data extraction, and logic reasoning.
    The platform is postulated as the central contribution but no implementation or external validation is described.

pith-pipeline@v0.9.1-grok · 5757 in / 1432 out tokens · 22930 ms · 2026-06-27T20:12:27.173685+00:00 · methodology

0 comments
read the original abstract

Solving machine learning problems is complex and typically reserved for experts. Over the past two decades, systems have emerged to support non-experts. Based on our review, we identify three categories: (1) fully automated AutoML systems, (2) expert cheat sheets for algorithm selection, and (3) decision-support systems using selection criteria (accuracy, transparency, data requirements). We propose a new platform combining categories 2 and 3 to deliver semi-automated, intelligent solution recommendations for non-experts. Unlike existing approaches that recommend a single algorithm, our platform suggests a complete pipeline tailored to the user's problem. It integrates expert-defined selection criteria with transfer learning and automatically extracts data characteristics (e.g., class imbalance, missing values) from user-provided datasets. The platform uses first-order logic to reason over its knowledge base and recommends suitable algorithms ranked by relevance. It features a user-friendly interface and connects to a crowdsourcing platform for ML experts, ensuring continuous updates. The platform is built incrementally, allowing seamless integration of new algorithms, criteria, and domain knowledge. To our knowledge, this is the first free, publicly accessible online framework that systematically captures and operationalizes expert knowledge to guide non-experts in solving ML problems in a structured, transparent manner.

Figures

Figures reproduced from arXiv: 2606.08212 by Hafedh Mili, Lokman Saleh, Mounir Boukadoum.

Figure 1
Figure 1. Figure 1: The cheat sheets from Microsoft, SAS, and Scikit-learn. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Machine learning algorithm selection procedure. [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Domain problem page. 3.4.2 Data Characteristics On the data characteristics page, the user must grant the platform access to their database. We offer two solutions: • Upload an Excel file. • Connect to a relational database. We focus on the second solution as it is most common in businesses. To connect to a remote relational database, the user enters the server authentication data (database name, user name… view at source ↗
Figure 4
Figure 4. Figure 4: Database schema. Next, the user specifies whether there is an attribute that is the answer (class attribute) to be predicted. This is done using a drop-down list ( [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Response attributes. By clicking “Create query”, the system automatically generates an SQL query that joins the selected attributes. A sample of 100 records, database features, and data-related selection criteria are displayed in separate tabs. 3.4.3 Data Analysis Problem The problem type (e.g., supervised vs. unsupervised) is automatically determined using data criteria from the previous step and user req… view at source ↗
Figure 6
Figure 6. Figure 6: Data analysis problem page. 3.4.4 Draft Solution Based on all the information entered, automatically extracted, or deduced, the processing chain is selected, reconstructed, or improved, including the choice of learning algorithm in the “Build model” component [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Draft solution page. The two platforms (icontributetoml.org and isolvemymlproblem.org) share three ta￾bles: users, selection criteria, and learning algorithms. This allows them to benefit from each other and synchronize data, making our solution scalable. New selection criteria or algorithms added by experts become immediately available. Synchronization between the platforms is performed by the website adm… view at source ↗
Figure 8
Figure 8. Figure 8: Naive Bayes obtained the highest score using the dot product. [PITH_FULL_IMAGE:figures/full_fig_p013_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

42 extracted references · 4 canonical work pages · 3 internal anchors

  1. [1]

    D-smartml: A distributed automated machine learning framework

    Ahmed Abd Elrahman, Mohamed El Helw, Radwa Elshawi, and Sherif Sakr. D-smartml: A distributed automated machine learning framework. In2020 IEEE 40th International Conference on Distributed Computing Systems (ICDCS), pages 1215–1218. IEEE

  2. [2]

    A new look at the statistical model identification.IEEE transactions on automatic control, 19(6):716–723, 1974

    Hirotugu Akaike. A new look at the statistical model identification.IEEE transactions on automatic control, 19(6):716–723, 1974

  3. [4]

    A roadmap of clustering algorithms: finding a match for a biomedical application.Briefings in Bioinfor- matics, 10(3):297–314, 2009

    Bill Andreopoulos, Aijun An, Xiaogang Wang, and Michael Schroeder. A roadmap of clustering algorithms: finding a match for a biomedical application.Briefings in Bioinfor- matics, 10(3):297–314, 2009

  4. [5]

    A survey on machine-learning techniques in cognitive radios.IEEE Communications Surveys & Tutorials, 15(3):1136– 1159, 2012

    Mario Bkassiny, Yang Li, and Sudharman K Jayaweera. A survey on machine-learning techniques in cognitive radios.IEEE Communications Surveys & Tutorials, 15(3):1136– 1159, 2012. 14

  5. [6]

    Au- tostacker: A compositional evolutionary learning system

    Boyuan Chen, Harvey Wu, Warren Mo, Ishanu Chattopadhyay, and Hod Lipson. Au- tostacker: A compositional evolutionary learning system. InProceedings of the Genetic and Evolutionary Computation Conference, pages 402–409

  6. [7]

    Au- tostacker: A compositional evolutionary learning system

    Boyuan Chen, Harvey Wu, Warren Mo, Ishanu Chattopadhyay, and Hod Lipson. Au- tostacker: A compositional evolutionary learning system. InProceedings of the Genetic and Evolutionary Computation Conference, pages 402–409, 2018

  7. [8]

    A survey on machine learning: concept, algo- rithms and applications.International Journal of Innovative Research in Computer and Communication Engineering, 5(2):1301–1309, 2017

    Kajaree Das and Behera Rabi Narayan. A survey on machine learning: concept, algo- rithms and applications.International Journal of Innovative Research in Computer and Communication Engineering, 5(2):1301–1309, 2017

  8. [9]

    A comparison of multiple classification methods for diagnosis of parkinson disease.Expert Systems with Applications, 37(2):1568–1572, 2010

    Resul Das. A comparison of multiple classification methods for diagnosis of parkinson disease.Expert Systems with Applications, 37(2):1568–1572, 2010

  9. [10]

    Recipe: a grammar-based framework for automatically evolving classification pipelines

    Alex GC de S´ a, Walter Jos´ e GS Pinto, Luiz Otavio VB Oliveira, and Gisele L Pappa. Recipe: a grammar-based framework for automatically evolving classification pipelines. In European Conference on Genetic Programming, pages 246–261. Springer

  10. [12]

    AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data

    Nick Erickson, Jonas Mueller, Alexander Shirkov, Hang Zhang, Pedro Larroy, Mu Li, and Alexander Smola. Autogluon-tabular: Robust and accurate automl for structured data. arXiv preprint arXiv:2003.06505, 2020

  11. [14]

    Survey of machine learning algorithms for disease diagnostic.Journal of Intelligent Learning Systems and Applications, 9(01):1, 2017

    Meherwar Fatima and Maruf Pasha. Survey of machine learning algorithms for disease diagnostic.Journal of Intelligent Learning Systems and Applications, 9(01):1, 2017

  12. [15]

    Multi-interval discretization of continuous-valued attributes for classification learning

    Usama M Fayyad and Keki B Irani. Multi-interval discretization of continuous-valued attributes for classification learning. InIjcai, volume 93, pages 1022–1029. Citeseer, 1993

  13. [16]

    Practical automated machine learning for the automl challenge 2018

    Matthias Feurer, Katharina Eggensperger, Stefan Falkner, Marius Lindauer, and Frank Hutter. Practical automated machine learning for the automl challenge 2018. InInterna- tional Workshop on Automatic Machine Learning at ICML, pages 1189–1232

  14. [17]

    Practical automated machine learning for the automl challenge 2018

    Matthias Feurer, Katharina Eggensperger, Stefan Falkner, Marius Lindauer, and Frank Hutter. Practical automated machine learning for the automl challenge 2018. InInterna- tional Workshop on Automatic Machine Learning at ICML, pages 1189–1232, 2018

  15. [18]

    Auto-sklearn: efficient and robust automated machine learning

    Matthias Feurer, Aaron Klein, Katharina Eggensperger, Jost Tobias Springenberg, Manuel Blum, and Frank Hutter. Auto-sklearn: efficient and robust automated machine learning. Automated Machine Learning, pages 113–134, 2018

  16. [19]

    P4ml: A phased performance-based pipeline planner for automated machine learning

    Yolanda Gil, Ke-Thia Yao, Varun Ratnakar, Daniel Garijo, Greg Ver Steeg, Pedro Szekely, Rob Brekelmans, Mayank Kejriwal, Fanghao Luo, and I-Hui Huang. P4ml: A phased performance-based pipeline planner for automated machine learning. InAutoML Workshop at ICML

  17. [20]

    Pspso: A package for parameters selection using particle swarm optimization.SoftwareX, 15:100706, 2021

    Ali Haidar, Matthew Field, Jonathan Sykes, Martin Carolan, and Lois Holloway. Pspso: A package for parameters selection using particle swarm optimization.SoftwareX, 15:100706, 2021. 15

  18. [21]

    The determination of the order of an autoregression

    Edward J Hannan and Barry G Quinn. The determination of the order of an autoregression. Journal of the Royal Statistical Society: Series B (Methodological), 41(2):190–195, 1979

  19. [22]

    Supervised machine learning: A re- view of classification techniques.Emerging artificial intelligence applications in computer engineering, 160(1):3–24, 2007

    Sotiris B Kotsiantis, I Zaharakis, and P Pintelas. Supervised machine learning: A re- view of classification techniques.Emerging artificial intelligence applications in computer engineering, 160(1):3–24, 2007

  20. [23]

    Auto-WEKA: Automatic model selection and hyperparameter optimization in WEKA, pages 81–95

    Lars Kotthoff, Chris Thornton, Holger H Hoos, Frank Hutter, and Kevin Leyton-Brown. Auto-WEKA: Automatic model selection and hyperparameter optimization in WEKA, pages 81–95. Springer, Cham, 2019

  21. [24]

    H2o automl: Scalable automatic machine learning

    Erin LeDell and Sebastien Poirier. H2o automl: Scalable automatic machine learning. In Proceedings of the AutoML Workshop at ICML, volume 2020

  22. [25]

    Smartml: A meta learning-based framework for auto- mated selection and hyperparameter tuning for machine learning algorithms

    Mohamed Maher and Sherif Sakr. Smartml: A meta learning-based framework for auto- mated selection and hyperparameter tuning for machine learning algorithms. InEDBT: 22nd International Conference on Extending Database Technology

  23. [26]

    Metwalli

    Sara A. Metwalli. How to choose the right machine learning al- gorithm for your application.https://towardsdatascience.com/ how-to-choose-the-right-machine-learning-algorithm-for-your-application-1e36c32400b9,

  24. [27]

    Consult´ e le 6 juin 2026

  25. [28]

    Machine learning algorithm cheat sheet for azure machine learning designer

    Microsoft. Machine learning algorithm cheat sheet for azure machine learning designer. 2020

  26. [29]

    Ml-plan: Automated machine learning via hierarchical planning.Machine Learning, 107(8):1495–1515, 2018

    Felix Mohr, Marcel Wever, and Eyke H¨ ullermeier. Ml-plan: Automated machine learning via hierarchical planning.Machine Learning, 107(8):1495–1515, 2018

  27. [31]

    In- ternational journal of innovative research in computer and communication engineering

    Ms Gaurangi Patil, Ms Varsha Galande, Mr Vedant Kekan, and Ms Kalpana Dange. In- ternational journal of innovative research in computer and communication engineering. Sentiment analysis using support vector machine, 2(1):2607–2612, 2014

  28. [32]

    DarwinML: A Graph-based Evolutionary Algorithm for Automated Machine Learning

    Fei Qi, Zhaohui Xia, Gaoyang Tang, Hang Yang, Yu Song, Guangrui Qian, Xiong An, Chunhuan Lin, and Guangming Shi. Darwinml: A graph-based evolutionary algorithm for automated machine learning.arXiv preprint arXiv:1901.08013, 2018

  29. [33]

    Automated machine learn- ing with monte-carlo tree search.arXiv preprint arXiv:1906.00170, 2019

    Herilalaina Rakotoarison, Marc Schoenauer, and Mich` ele Sebag. Automated machine learn- ing with monte-carlo tree search.arXiv preprint arXiv:1906.00170, 2019

  30. [34]

    A quick review of machine learning algorithms

    Susmita Ray. A quick review of machine learning algorithms. In2019 International con- ference on machine learning, big data, cloud and parallel computing (COMITCon), pages 35–39. IEEE

  31. [35]

    Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks

    Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks.arXiv preprint arXiv:1908.10084, 2019

  32. [36]

    How to select a suitable machine learning algorithm: A feature-based, scope-oriented selection framework

    R Sala, M Zambetti, F Pirola, and R Pinto. How to select a suitable machine learning algorithm: A feature-based, scope-oriented selection framework. In23rd Summer School” Francesco Turco”-Industrial Systems Engineering 2018, volume 2018, pages 87–93. AIDI- Italian Association of Industrial Operations Professors. 16

  33. [38]

    A review of clustering techniques and developments.Neurocomputing, 267:664–681, 2017

    Amit Saxena, Mukesh Prasad, Akshansh Gupta, Neha Bharill, Om Prakash Patel, Aruna Tiwari, Meng Joo Er, Weiping Ding, and Chin-Teng Lin. A review of clustering techniques and developments.Neurocomputing, 267:664–681, 2017

  34. [39]

    Estimating the dimension of a model.The annals of statistics, 6(2):461– 464, 1978

    Gideon Schwarz. Estimating the dimension of a model.The annals of statistics, 6(2):461– 464, 1978

  35. [40]

    Machine learning map.https://scikit-learn.org/stable/ tutorial/machine_learning_map/index.html, 2020

    scikit-learn developers. Machine learning map.https://scikit-learn.org/stable/ tutorial/machine_learning_map/index.html, 2020. Consult´ e le 6 juin 2026

  36. [41]

    Zoomed ranking: Selection of classification algorithms based on relevant performance information

    Carlos Soares and Pavel B Brazdil. Zoomed ranking: Selection of classification algorithms based on relevant performance information. InEuropean conference on principles of data mining and knowledge discovery, pages 126–135. Springer

  37. [42]

    Yoel S´ anchez Berm´ udez.How to select the right machine learning approach?Degree project, 2013

  38. [43]

    Guidelines to select machine learning scheme for classification of biomedical datasets

    Ajay Kumar Tanwani, Jamal Afridi, M Zubair Shafiq, and Muddassar Farooq. Guidelines to select machine learning scheme for classification of biomedical datasets. InEuropean Conference on Evolutionary Computation, Machine Learning and Data Mining in Bioin- formatics, pages 128–139. Springer

  39. [44]

    Auto-weka: Combined selection and hyperparameter optimization of classification algorithms

    Chris Thornton, Frank Hutter, Holger H Hoos, and Kevin Leyton-Brown. Auto-weka: Combined selection and hyperparameter optimization of classification algorithms. InPro- ceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 847–855

  40. [45]

    Automated machine learning in practice: state of the art and recent results

    Lukas Tuggener, Mohammadreza Amirian, Katharina Rombach, Stefan L¨ orwald, Anastasia Varlet, Christian Westermann, and Thilo Stadelmann. Automated machine learning in practice: state of the art and recent results. In2019 6th Swiss Conference on Data Science (SDS), pages 31–36. IEEE

  41. [46]

    Hyperparameter importance across datasets

    Jan N Van Rijn and Frank Hutter. Hyperparameter importance across datasets. InPro- ceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2367–2376, 2018

  42. [47]

    A novel evolutionary algorithm for automated machine learning focusing on classifier ensem- bles

    Joao C Xavier-J´ unior, Alex A Freitas, Antonino Feitosa-Neto, and Teresa B Ludermir. A novel evolutionary algorithm for automated machine learning focusing on classifier ensem- bles. In2018 7th Brazilian Conference on Intelligent Systems (BRACIS), pages 462–467. IEEE. 17