Comparison of Artificial Intelligence Techniques for Project Conceptual Cost Prediction

Haytham H. Elmousalami

REVIEW 6 major objections 6 minor 5 references

Comparison of Artificial Intelligence Techniques for Project Conceptual Cost Prediction

T0 review · 6 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read XGBoost beats 19 AI models at predicting early-stage project costs, with 9.091% MAPE on a 144-project irrigation dataset.

desk verdict The XGBoost win is almost certainly within noise due to a single-split evaluation, but the benchmark is a useful practical survey if properly re-validated. read the letter →

arxiv 1909.11637 v1 pith:A76F5454 submitted 2019-08-08 cs.LG cs.AI

Haytham H. Elmousalami This is my paper · ORCID

classification cs.LGcs.AI

keywords conceptualcostpredictionXGBoostensemblemethodsmachinelearningconstructionestimationfuzzylogicgeneticparametricmodel

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that, among twenty supervised regression techniques applied to a real-world dataset of 144 Egyptian field canal improvement projects, XGBoost gives the most accurate conceptual cost estimates, with 9.091% MAPE and adjusted R-squared 0.929. This matters because conceptual cost estimates are made early in a project with little information, and practitioners need guidance on which method to trust. The paper also argues that ensemble methods outperform single models, that data transformations help regression and neural models, and that fuzzy models trade accuracy for uncertainty handling. The comparison itself is the contribution: a single benchmark ranking on one dataset, not a proof of general superiority.

What carries the argument

The central object is the XGBoost ensemble: an additive model $\hat{y}_i = \sum_k f_k(x_i)$ of regression trees trained with a regularized objective $L = \sum_k \ell(\hat{y}_k, y_k) + \sum_k \Omega(f_k)$, where $\Omega(f) = \gamma T + \frac{1}{2}\lambda \|w\|^2$. The regularization term and default missing-value handling are what the paper credits for XGBoost's top ranking; the comparison table and the MAPE and adjusted R-squared metrics carry the argument.

What would settle it

Run the same models on the same FCIP data with repeated stratified k-fold cross-validation and record the variance of MAPE. If the cross-validated mean gap between XGBoost and quadratic regression is not reliably positive, or if feature selection is done inside each fold and the ranking flips, the paper's central claim is not supported.

Watch

Extended reading notes

Core claim

The central claim is that XGBoost is the most accurate and suitable AI technique for conceptual cost prediction on the FCIP dataset, edging out quadratic regression at 9.120% MAPE and plain regression at 9.130%, and clearly beating DNNs at 12.059%, decision trees at 12.488%, genetic-fuzzy at 14.700%, case-based reasoning at 17.300%, SVM at 21.217%, and expert fuzzy logic at 26.300%. The paper presents this as evidence that scalable tree boosting, with its built-in regularization and missing-value handling, is the appropriate default model for small, noisy project-cost data.

Load-bearing premise

The ranking depends on one random split of 144 projects into 111 training and 33 test cases, and the top two models are within 0.03 percentage points, so a different split could change the winner.

Editorial extensions

If this is right

If the ranking holds, project managers estimating early-stage irrigation improvement costs should prefer XGBoost over the previously recommended quadratic regression, at least for similar FCIP-type data.
Ensemble methods (Extra Trees, bagging, random forest, AdaBoost, stochastic gradient boosting) all land below 11% MAPE, supporting ensemble approaches over single learners when data are small and noisy.
The gap between genetic-fuzzy at 14.7% and expert fuzzy at 26.3% implies that automated fuzzy rule generation is the route to making uncertainty-handling fuzzy models usable in cost estimation.
DNNs at 12.059% MAPE were worse than every ensemble method, indicating that deep networks are not suitable for datasets of this size.
The paper supplies a benchmark ranking that future conceptual cost prediction studies can compare against when choosing or proposing new models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The margin between first and second place is 0.029 percentage points, far smaller than the uncertainty of a single random split; repeated k-fold cross-validation could easily reorder the top models.
The feature selection was performed on the full dataset before splitting, so information from the test cases may have leaked into model construction and the reported test MAPEs could be optimistic.
A testable extension is to run the same twenty-model comparison on other construction project types and report cross-validated means and variances for each model rather than single-split scores.
The paper's uncertainty-handling argument points toward a natural follow-up: a hybrid that combines XGBoost accuracy with fuzzy interval outputs, giving both accuracy and uncertainty bounds.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

XGBoost is almost certainly not the winner here. The paper compares 20 AI methods on 144 Egyptian field canal projects and finds XGBoost best by a hair (9.091% vs 9.120% MAPE for quadratic regression), but a single random split of 111/33 with no cross-validation or significance testing means the top five finishers are statistically indistinguishable. That's the main thing to know.

Credit where it's due: the paper is a genuine benchmark, not a toy. The author applies a broad set of techniques—regression, ANNs, DNNs, CBR, fuzzy, genetic-fuzzy, SVM, and several ensemble methods—to a real cost-estimation problem, and the comparison on this specific FCIP dataset hasn't been published before. The text is organized around a practical question, and there's an honest discussion of model interpretability, uncertainty, and small-sample issues. The author also correctly notes that his earlier work had quadratic regression as the best model; he's not hiding the fact that the winner label is unstable.

The soft spots, in order of severity: (1) The evaluation protocol. One random split gives no error bars. With 33 test points, the standard error of MAPE is on the order of 1–2 percentage points, so a 0.029-point gap between M1 and M2 is meaningless. No paired test, no bootstrap, no repeated CV. (2) Feature selection leakage. The four cost drivers were selected by Pearson correlation and stepwise regression on the full 144-project dataset in Elmousalami et al. 2018b, so those features already encode test-set information. That inflates absolute accuracy and can distort the ranking. (3) Missing reproducibility. No code, no data, no hyperparameter settings for the 20 models. The fuzzy and GA portions are described only in outline. (4) Minor typos: Eq. 4 has an inequality that doesn't make sense, Eq. 8 writes the XGBoost objective as $(x+a)^n$ (a binomial expansion, not the actual loss), and the conclusion says 21 models while Table 1 lists 20.

These are not fatal to the paper's existence, but they are fatal to the headline claim as stated. The comparison itself is not circular—the XGBoost result is empirical, not derived from its inputs—and the author's engagement with the prior literature is honest. But the practical recommendation to cost engineers ('use XGBoost') is not supported by this evidence.

Who gets value: a cost engineer wanting a quick map of what AI methods exist and roughly how they do on this kind of data, or a graduate student looking for a case study in why single-split benchmarks are dangerous. The paper is not a research contribution that will change methodology.

If I were the editor, I'd send it to peer review only with a strong directive: rerun with cross-validation or bootstrap, add significance tests, release the data and hyperparameters, and fix the leakage issue. If the author can do that, a revised version could be useful. As it stands, I'd treat the XGBoost recommendation as unverified.

Referee Report

6 major / 6 minor

Summary. The paper compares twenty AI and machine-learning techniques for conceptual cost prediction of field canals improvement projects (FCIPs) in Egypt. Using a dataset of 144 projects split into 111 training and 33 test cases, the authors evaluate models from multiple families (regression, ANNs/DNNs, SVM, fuzzy and genetic-fuzzy systems, CBR, decision trees, and ensemble methods) with MAPE and adjusted R^2. The headline claim is that XGBoost is the most accurate and suitable model, with 9.091% MAPE and 0.929 adjusted R^2 (Table 1, M1), narrowly ahead of quadratic regression at 9.120% MAPE. The paper also discusses qualitative properties such as interpretability, uncertainty handling, and robustness to noisy data, and provides a strengths/weaknesses table for all models.

Significance. If the reported ranking were robust, the paper would provide useful practical guidance for cost-estimation practitioners and a structured benchmark across many model families. The study is creditable for assembling twenty models on a real dataset, for giving a transparent table of model characteristics, and for explicitly discussing the uncertainty/interpretability trade-off. However, the significance is limited by the evaluation methodology: the ranking rests on a single random split with no significance testing, and several reported metrics appear internally inconsistent. These issues are fixable in revision, but as it stands the central claim is not statistically supported.

major comments (6)

[Comparison and analysis, Table 1] The headline claim that XGBoost is the most accurate model is not statistically supported. The evaluation uses a single random split (111/33) with no cross-validation, bootstrap, or paired significance test. The top two models differ by only 0.029 percentage points in MAPE (9.091% vs. 9.120%), while the top five span 9.091–9.270%. With 33 test points the standard error of MAPE is plausibly on the order of 1–2 percentage points, so the observed differences are within noise. Please report repeated cross-validation results and a paired test (e.g., Wilcoxon or paired t-test) of the XGBoost-vs-quadratic-regression difference, or explicitly temper the conclusion to 'performs well on this split'.
[Data collection and feature selection] Feature selection leaks test information into every model. The four cost drivers were selected by Pearson correlation and stepwise regression on the full 144-case dataset in Elmousalami et al. (2018b), and the same dataset is then split for training and testing. This can inflate absolute accuracy and distort the ranking because the preselected features were chosen partly from test-case information. The prior study's finding that quadratic regression was 'the most accurate' on the same data further indicates that the winner label is unstable across studies. Please use a nested procedure (e.g., feature selection inside each cross-validation fold) or at least discuss this limitation explicitly.
[Table 1, Eqs. (9)–(10)] Several reported adjusted R^2 values exceed the corresponding R^2 values (e.g., M8: R2=0.905, R*2=0.910; M5: R2=0.913, R*2=0.912). By the definition in Eq. (10), adjusted R^2 cannot exceed R^2 when K>0, so these entries are internally inconsistent and indicate computational or rounding errors in the metric calculations. Please recheck the reported adjusted R^2 values and the underlying formulas; the ranking by R*2 may change.
[Support Vector Machines, Eq. (4)] Equation (4), y_i(W·X_i+b) ≥ 0 − ξ, is a mis-specified classification constraint, not the support vector regression formulation the paper claims to apply. The subsequent objective in Eq. (5) also corresponds to classification-style soft-margin SVM, not to ε-insensitive SVR. Please correct the mathematical description of the SVM model, or remove the equations if the model was fitted with a standard library using default settings.
[Extreme Gradient Boosting, Eq. (8)] Equation (8) is garbled: it begins with the binomial expansion identity (x+a)^n = ..., which has no connection to the XGBoost objective, and then mixes a loss term and a regularization term without a clear definition of the components. The intended XGBoost loss with regularization is standard; please rewrite Eq. (8) properly so the description is technically accurate.
[Genetic-Fuzzy model] There is a numerical inconsistency in the fuzzy rule count. The text says 'the number of possible rules equals 74 rules' while later stating '2401 rules represent the whole possible search space for GA.' With four inputs and seven membership functions per input, the correct number is 7^4 = 2401. The '74' appears to be a typo for '7^4,' but as written it is confusing; please correct it.

minor comments (6)

[Abstract and Conclusion] The abstract says 'twenty artificial intelligence techniques' but the conclusion says 'Twenty one machine learning models are developed'; please make the count consistent (Table 1 lists 20).
[Evaluation techniques] The MAPE categorization equation is hard to parse because of missing percentage signs and ambiguous inequalities; for clarity, rewrite as explicit categories (e.g., MAPE < 10% excellent, 10% ≤ MAPE < 20% good, MAPE ≥ 20% unacceptable).
[MRA and transformed regressions] The text refers to 'Haytham et al. (2018 b)' but the reference list uses 'Elmousalami et al. 2018b'; please standardize the author name throughout.
[References] Several references are duplicated (Fan et al. 2008, Schapire et al. 1998, Stoy et al. 2012); please remove duplicates.
[Model development sections] Hyperparameter values are only reported for the genetic algorithm (crossover 0.7, mutation 0.01). Please report the hyperparameters used for SVM (C, gamma/kernel parameters), XGBoost (learning rate, max depth, number of trees, etc.), and the DNN (optimizer, epochs, batch size) to make the comparisons reproducible.
[Fig. 7] Figure 7 plots MAPE and R^2 with different scales on separate subfigures; consider adding clear axis labels and possibly a combined figure with normalized scales for easier visual comparison across models.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the XGBoost ranking is an empirical comparison result, not a derivation from fitted inputs or a self-citation chain.

full rationale

The paper's claimed result is an empirical ranking: XGBoost achieves the lowest MAPE (9.091%) and adjusted R2 (0.929) among twenty models on a fixed 111/33 split of the FCIP data (Table 1). Nothing in the construction of XGBoost, the fuzzy fitness function, or the ensemble equations defines this ranking in terms of the input features or the competing models; the winning score is measured, not derived from the feature-selection step or from the authors' earlier publications. The self-citations to Elmousalami et al. (2018a, 2018b) supply the dataset, the four key cost drivers, and seven baseline models, but those are inputs and competitors rather than the conclusion; the XGBoost performance is not a restatement of any of those cited results. The absence of cross-validation and the use of a single random split are legitimate methodological concerns, as is the fact that feature selection was performed before the split, but these are statistical robustness and leakage risks, not circularity: no fitted parameter is renamed as a prediction, and no equation reduces to another by construction. Accordingly, no circular step is exhibited.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The central claim is an empirical ranking, so it does not rely on new mathematical axioms. The main assumptions are inherited: the four cost drivers are sufficient, the single random split is representative, and the benchmark models from the authors' earlier work are correctly reused. Hyperparameters and design choices for several models are not reported, which is reflected in the free parameters list.

free parameters (7)

CBR attribute weights (AW)
Selected by an expert in the CBR model (Section 'Case-based reasoning (CBR)'); no values or sensitivity analysis are given.
Fuzzy membership function parameters
Seven triangle MFs per input and output are chosen without reporting their centers or widths (Section 'Fuzzy logic model').
GA crossover and mutation probabilities = 0.7 and 0.01
Set by hand in the Genetic-Fuzzy model (Section 'Genetic-Fuzzy model'); no tuning or sensitivity analysis is reported.
Expert fuzzy rule set = 190 rules
The traditional fuzzy model uses 190 expert-formulated IF-Then rules; the rule set is a hand-crafted input (Section 'Fuzzy logic model').
SVM hyperparameters (C, gamma)
RBF kernel is specified but penalty and kernel parameters are not reported (Section 'Support Vector Machines').
XGBoost hyperparameters
The winning model's learning rate, tree depth, number of estimators, etc. are not reported (Section 'Extreme Gradient Boosting').
DNN architecture = 4-100-100-100-1
Three hidden layers of 100 neurons with ReLU are chosen without justification (Section 'ANNs and DNNs').

assumptions (5)

domain assumption The four key cost drivers (area served, pipeline total length, number of irrigation valves, construction year) are sufficient to predict conceptual cost.
Feature selection was performed in prior work by the same authors on the same dataset; this paper inherits the choice without re-validation (Section 'Data collection and feature selection').
domain assumption The random 111/33 split of the 144 projects gives an unbiased estimate of generalization.
Only one split is used; no cross-validation or stratification is reported, so this assumption is untested (Section 'Data collection and feature selection').
standard math MAPE and adjusted R-squared are appropriate and sufficient metrics for model comparison.
These are standard metrics, but the paper does not test whether the observed differences are statistically significant (Section 'Evaluation techniques').
domain assumption Benchmark models M2-M6, M8, M13, M14 from Elmousalami et al. (2018b) were correctly transferred to this comparison.
The paper reuses these models without describing re-fitting details or verifying that the comparison is fair (Section 'Comparison and analysis').
standard math Green's rule [50 + 8*N] justifies the 111-sample training set.
Heuristic sample-size rule used in Section 'Data collection and feature selection'; it does not guarantee sufficient power for 20 models.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Comparison of Artificial Intelligence Techniques for Project Conceptual Cost Prediction." pith.science (2026). https://pith.science/paper/A76F5454

@misc{pith2026190911637,
  author       = {Pith},
  title        = {Pith review of: Comparison of Artificial Intelligence Techniques for Project Conceptual Cost Prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/A76F5454}},
  note         = {Machine review of arXiv:1909.11637}
}

read the original abstract

Developing a reliable parametric cost model at the conceptual stage of the project is crucial for projects managers and decision-makers. Existing methods, such as probabilistic and statistical algorithms have been developed for project cost prediction. However, these methods are unable to produce accurate results for conceptual cost prediction due to small and unstable data samples. Artificial intelligence (AI) and machine learning (ML) algorithms include numerous models and algorithms for supervised regression applications. Therefore, a comparison analysis for AI models is required to guide practitioners to the appropriate model. The study focuses on investigating twenty artificial intelligence (AI) techniques which are conducted for cost modeling such as fuzzy logic (FL) model, artificial neural networks (ANNs), multiple regression analysis (MRA), case-based reasoning (CBR), hybrid models, and ensemble methods such as scalable boosting trees (XGBoost). Field canals improvement projects (FCIPs) are used as an actual case study to analyze the performance of the applied ML models. Out of 20 AI techniques, the results showed that the most accurate and suitable method is XGBoost with 9.091% and 0.929 based on Mean Absolute Percentage Error (MAPE) and adjusted R2. Nonlinear adaptability, handling missing values and outliers, model interpretation and uncertainty have been discussed for the twenty developed AI models. Keywords: Artificial intelligence, Machine learning, ensemble methods, XGBoost, evolutionary fuzzy rules generation, Conceptual cost, and parametric cost model.

Figures

Figures reproduced from arXiv: 1909.11637 by the authors.

**Figure 2.** Fuzzy rules firing. The first step in the FL model is fuzzification the four key cost drivers and identify their MFs. The most critical stage is to develop fuzzy rules base. experts are consulted to give their experience to develop such rules. As shown in [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

5 extracted references · 4 canonical work pages

[4]

The annals of statistics, 26(5), pp.1651-1686

Boosting the margin: A new explanation for the effectiveness of voting methods. The annals of statistics, 26(5), pp.1651-1686. Schapire, R.E., Freund, Y., Bartlett, P. and Lee, W.S.,

work page
[5]

A concept for developing construction element cost models for German residential building projects

Boosting the margin: A new explanation for the 14 effectiveness of voting methods. The annals of statistics, 26(5), pp.1651-1686. Shreenaath.A, Arunmozhi.S., Sivagamasundari.R,(2015),Prediction of Construction Cost Overrun in Tamil Nadu - A Statistical Fuzzy Approach,International Journal of Engineering and Technical Research (IJETR) Siddique, N., and Ade...

work page 2015
[1984]

Evolutionary fuzzy decision model for construction management using support vector machine

Classification and Regression Trees. California, Wadsworth. Chen, T. and Guestrin, C., 2016, August. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining (pp. 785 -794). ACM. Cheng, M. -Y., and Roy, A. F. 2010. “Evolutionary fuzzy decision model for construction man...

arXiv 2006
[1989]

What is computational intelligence? In Computational Intelligence Imitating Life

pp. 144-147. Bauer, E. and Kohavi, R., 1999. An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine learning, 36(1 -2), pp.105-139. Berry, M.J. and Linoff, G., 1997. Data mining techniques: for marketing, sales, and customer support. John Wiley & Sons, Inc.. Bezdek, J.C. 1994. “What is computational intellige...

work page 1999
[1998]

Predicting telecommunication tower costs using fuzzy subtractive clustering

Forecasting methods and applic ations, New York, Wiley. Marzouk, M., and Alaraby, M. 2014. “Predicting telecommunication tower costs using fuzzy subtractive clustering.” Journal of Civil Engineering and Management, 21(1), 67–74. Marzouk, M., and Elkadi, M. (2016). “Estimating water treatment plants costs using factor analysis and artificial neural network...

work page 2016

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [4]

The annals of statistics, 26(5), pp.1651-1686

Boosting the margin: A new explanation for the effectiveness of voting methods. The annals of statistics, 26(5), pp.1651-1686. Schapire, R.E., Freund, Y., Bartlett, P. and Lee, W.S.,

work page

[2] [5]

A concept for developing construction element cost models for German residential building projects

Boosting the margin: A new explanation for the 14 effectiveness of voting methods. The annals of statistics, 26(5), pp.1651-1686. Shreenaath.A, Arunmozhi.S., Sivagamasundari.R,(2015),Prediction of Construction Cost Overrun in Tamil Nadu - A Statistical Fuzzy Approach,International Journal of Engineering and Technical Research (IJETR) Siddique, N., and Ade...

work page 2015

[3] [1984]

Evolutionary fuzzy decision model for construction management using support vector machine

Classification and Regression Trees. California, Wadsworth. Chen, T. and Guestrin, C., 2016, August. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining (pp. 785 -794). ACM. Cheng, M. -Y., and Roy, A. F. 2010. “Evolutionary fuzzy decision model for construction man...

arXiv 2006

[4] [1989]

What is computational intelligence? In Computational Intelligence Imitating Life

pp. 144-147. Bauer, E. and Kohavi, R., 1999. An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine learning, 36(1 -2), pp.105-139. Berry, M.J. and Linoff, G., 1997. Data mining techniques: for marketing, sales, and customer support. John Wiley & Sons, Inc.. Bezdek, J.C. 1994. “What is computational intellige...

work page 1999

[5] [1998]

Predicting telecommunication tower costs using fuzzy subtractive clustering

Forecasting methods and applic ations, New York, Wiley. Marzouk, M., and Alaraby, M. 2014. “Predicting telecommunication tower costs using fuzzy subtractive clustering.” Journal of Civil Engineering and Management, 21(1), 67–74. Marzouk, M., and Elkadi, M. (2016). “Estimating water treatment plants costs using factor analysis and artificial neural network...

work page 2016