Pith. sign in

REVIEW 5 major objections 5 minor 1 cited by

Enhancing Project Performance Forecasting using Machine Learning Techniques

T0 review · 5 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Machine-learning time-series models, especially LSTM, can forecast project cost variance and earned value more accurately than traditional earned value management in a road reconstruction case study.

desk verdict Standard ARIMA/LSTM case study whose central LSTM-vs-ARIMA claim is unsupported by any reported numbers and whose external-factor findings are artifacts of simulated inputs. read the letter →

arxiv 2411.17914 v1 pith:5SRUKQUN submitted 2024-11-26 cs.LG cs.AIcs.CYstat.AP

Soheila Sadeghi This is my paper · ORCID
classification cs.LGcs.AIcs.CYstat.AP
keywords projectperformanceforecastingmachinelearningtimeseriesLSTMARIMAearnedvaluemanagementcostvarianceconstructionprojects
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to prove that machine-learning time series models can forecast project performance indicators better than the traditional earned value management (EVM) baseline. It trains ARIMA and LSTM models on historical cost and earned-value data from an urban road reconstruction project, adding simulated weather patterns and resource availability as external features. The case study claims LSTM produces the most accurate forecasts, with lower mean absolute error, mean squared error, and root mean squared error than ARIMA and EVM, and that feature-importance analysis shows cost history and planned value are the strongest drivers. If this holds, project managers gain an early-warning tool that flags deviations from baseline before they grow.

What carries the argument

The central mechanism is a per-Work Breakdown Structure (WBS) time series forecast of cost variance and earned value, built from engineered rolling averages and simulated external factors. ARIMA captures the linear trend component of the series, while LSTM, a recurrent neural network, captures long-term dependencies and nonlinear patterns; both are compared against the EVM baseline. SHAP (SHapley Additive exPlanations) values are used to attribute each feature's contribution to the predictions, and k-fold cross-validation provides the robustness check.

What would settle it

Retrain ARIMA and LSTM on the same project with recorded, not simulated, weather and resource data, and compute MAE, MSE, and RMSE on a held-out period; if LSTM's errors are not below ARIMA's and EVM's, the paper's central claim fails.

Watch

Extended reading notes

Core claim

The paper argues that the weakness of EVM is its reliance on static baseline plans, which ignore the dynamic, nonlinear patterns in project progress and external influences. By treating cost variance and earned value as time series and modeling them with ARIMA and LSTM, the forecast can adapt as new progress data arrives. In the road reconstruction case study, the LSTM model is reported to have the best performance on MAE, MSE, and RMSE, and cross-validation confirms the models generalize across data subsets; SHAP analysis identifies rolling average actual cost and rolling average planned value as the most influential features. The conclusion drawn is that machine learning, with external factors included, significantly improves project performance prediction.

Load-bearing premise

The load-bearing premise is that simulated weather and resource-availability values can stand in for real measurements of those factors; if the simulation is not representative of the actual project, the findings about external factors and the comparison with EVM do not necessarily hold for real conditions.

Editorial extensions

If this is right

  • Project managers could identify WBS categories drifting from baseline early and take corrective action before cost variance widens.
  • Collecting real weather and resource-availability data would make the same models directly usable for ongoing project monitoring.
  • The reported superiority of LSTM over ARIMA implies that nonlinear, long-memory patterns carry forecasting signal that linear models miss.
  • Feature-importance rankings can tell project teams which metrics deserve the most disciplined data collection.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper does not pursue is an online forecasting loop that retrains or updates the model as each progress payment and actual cost record arrives.
  • Because the external factors were simulated, the strongest test of the paper's message is to rerun the same models with recorded weather and resource data and compare the SHAP rankings.
  • The per-WBS granularity suggests the method would transfer to other infrastructure projects, but projects with short histories or irregular reporting may not have enough data to train an LSTM reliably.
  • If the findings hold, the practical bottleneck shifts from model choice to data quality: the value of the forecast depends on how faithfully cost and progress histories are recorded.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes a machine-learning-based forecasting approach for project performance metrics (cost variance, earned value) in an urban road reconstruction project, using ARIMA and LSTM time-series models and incorporating external factors (weather patterns, resource availability) as features. The authors report summary statistics and a correlation table, claim that both ARIMA and LSTM outperform traditional earned value management, and state that LSTM gives the most accurate forecasts. The case-study period, model-development details, and feature-importance analysis are described, but the central quantitative evidence—MAE/MSE/RMSE values and the figures that display them—does not appear in the manuscript.

Significance. If substantiated, the finding that LSTM-based forecasting outperforms both ARIMA and traditional EVM in a road reconstruction project would be a practically useful contribution to construction project management, especially with the inclusion of external-factor features and SHAP-based interpretability. However, the current manuscript does not provide the evidence needed to assess that claim: no numeric error metrics are reported, the referenced figures are absent, the external factors are simulated, and the cross-validation design is not described with enough specificity to rule out temporal leakage. The paper also provides no reproducible code or data release. The potential significance is therefore real but entirely unverified; the contribution at present is an architecture proposal and a set of claims rather than a validated empirical study.

major comments (5)
  1. [Section IV, Figs. 1–2] The central claim that “the LSTM model, in particular, showcased superior performance” and that both models “outperformed the traditional EVM approach” is not supported by any quantitative evidence in the text. Section IV refers to MAE, MSE, and RMSE and to Figs. 1 and 2, but the figures are not present in the manuscript and no numeric values of these metrics are given in any table or sentence. Table II reports only Pearson correlations between features, not forecast errors. Without the actual MAE/MSE/RMSE values, per-metric breakdowns, and ideally confidence intervals or significance tests, the comparative claim cannot be checked or reproduced.
  2. [Section III.A, Section III.D] The external factors are simulated: the text states that weather and resource data were “simulated based on reasonable assumptions to demonstrate the potential impact of external factors.” The paper then treats the SHAP feature-importance results and the correlation analysis as evidence that external factors matter for forecasting. This is circular: the simulated variables can only influence the models through the assumptions baked into their generation, so the findings about weather and resource availability are artifacts of the simulation unless the simulation is validated against real measurements or explicitly presented only as an illustrative sensitivity exercise. The manuscript does neither, and Section V repeats the claim that “the integration of simulated external factors… contributed to a more comprehensive and accurate assessment.”
  3. [Section III.C] The description of k-fold cross-validation is insufficient and potentially invalid for time-series data. The dataset spans only 22 months, yet the text does not state the value of k, the split ratio, or whether the folds are contiguous time blocks. Standard random k-fold splitting of a time series leaks future information into the training set, producing optimistically biased error estimates. Since the error estimates themselves are not reported, the claimed “confidence in the models’ predictive capabilities” is unsubstantiated. The authors should use walk-forward validation or an explicit temporal split and report fold-wise results.
  4. [Section III.B] The LSTM architecture is described only as “multiple layers” with “the number of neurons and layers determined through experimentation and hyperparameter tuning.” No layer counts, neuron counts, learning rate, batch size, sequence length, loss function, optimizer, or training epochs are given. Likewise, the ARIMA model is mentioned without its (p,d,q) orders. These details are load-bearing because the claimed superiority of LSTM may depend on specific hyperparameter choices, and the absence of such details makes the experiments impossible to reproduce or to assess for overfitting. At minimum, a table of hyperparameters and a description of the tuning procedure should be provided.
  5. [Section III.D, Section III.A] There is an inconsistency in the time periods: data collection is described as January 2011 to October 2012 in Section III.A, while the case study application is described as October 2011 to November 2012 in Section III.D. It is also unclear how the data are split into training, validation, and test sets, and what forecast horizon the models were asked to predict. Clarifying the exact temporal structure of the training and evaluation is necessary for the results to be meaningful.
minor comments (5)
  1. [Throughout] There are duplicated subsection headings (two “A.” subsections in Section III), inconsistent references to “Table 2” vs. “Table II,” and a repeated title header in the body text; these editorial issues should be fixed.
  2. [Section IV] The captions for Figs. 1–3 are present but the figures themselves are missing from the manuscript; either the figures must be included or the text should report the numerical values they would show.
  3. [Section II / References] Reference [8] is marked “submitted for publication” and reference [3] is marked “unpublished”; these are not citable sources in their current form, and several other references appear incomplete or in preprint form, which weakens the literature-review grounding.
  4. [Section III.A] The text states that “data preprocessing steps, including handling missing values, outliers, and inconsistencies” were applied, but no details are given about the amount of missing data, the outlier criteria, or the normalization method; such details are needed for reproducibility.
  5. [Section IV] The phrase “significant improvements” and “significantly improved” in the conclusion is not supported by any statistical significance testing; the authors should either provide tests with p-values or use language such as “substantially lower error” backed by the reported metrics.

Circularity Check

1 steps flagged · score 6.0 of 10

External-factor importance is an artifact of simulation; LSTM superiority is unverifiable but not circular.

  1. self definitional [Section III.A; Section IV (feature importance paragraph); Section V (conclusion)]
    "Due to the unavailability of external data on resource availability and weather conditions in the original dataset, these factors were simulated based on reasonable assumptions to demonstrate the potential impact of external factors on project performance forecasting. ... Similarly, the feature importance analysis highlighted the significant impact of external factors on project performance and predicted cost variance and earned value."

    The external factors are constructed under assumptions whose stated purpose is 'to demonstrate the potential impact'; the later SHAP/feature-importance analysis then 'highlighted the significant impact', and the conclusion credits 'simulated external factors' with improving forecasts. The 'discovery' is therefore an artifact of the simulation design, not an empirical result. Because no measured weather/resource data enter the pipeline, the feature-importance values for those variables reflect only the analyst's assumptions. The paper's own limitation statement, 'the assumptions made regarding the impact of external factors', admits that the simulated impact is assumed, making the external-factor conclusion equivalent to its input by construction.

full rationale

I identified one load-bearing circular step, not more. The LSTM/ARIMA/EVM performance comparison is central but unverifiable because the manuscript names MAE, MSE, and RMSE yet reports no numeric values and the referenced figures are absent; however, missing evidence is a correctness/verifiability problem, not circularity. Similarly, random k-fold splitting of a 22-month time series is a potential leakage risk, but no claim is shown to reduce by construction. The genuine circularity is the external-factor story: Section III.A simulates weather and resource availability 'to demonstrate the potential impact of external factors', and Section IV presents the feature-importance result as evidence that external factors matter. That conclusion is forced by the simulation assumptions, so the external-factor part of the claimed contribution reduces to its inputs. Because the paper also makes an independent (though unsubstantiated) forecasting-comparison claim that is not itself circular, a partial-circularity score of 6 is appropriate.

Assumptions & free parameters 4 free parameters · 3 assumptions · 2 invented entities

The central claim rests on several unverified inputs: unspecified model hyperparameters, rolling-average features whose windows are not given, and external-factor variables that were entirely simulated because real data were unavailable. The most consequential choice is the simulation of weather and resource availability, because every statement about the value of external factors depends on those synthetic numbers. The paper provides no independent data or formal checks to validate these inputs.

free parameters (4)
  • LSTM architecture hyperparameters (layers, neurons, learning rate, batch size, sequence length) = Not reported
    Section III.B says the architecture and training choices were determined through experimentation and hyperparameter tuning, but no final values are provided. All LSTM forecasts depend on these choices.
  • ARIMA model orders (p,d,q) = Not reported
    Section III.B mentions an ARIMA model but gives no order selection, estimation method, or diagnostic check, so the ARIMA forecasts cannot be reconstructed.
  • Rolling average window sizes = Not reported
    Section III.A introduces rolling averages of cost variance, planned value, earned value, and actual cost as features, but the window length is never stated.
  • Simulation parameters for weather and resource availability = Not reported
    Section III.A states that external factors were simulated based on 'reasonable assumptions' because real data were unavailable. The generated values are hand-chosen inputs that drive the external-factor conclusions.
assumptions (3)
  • domain assumption Historical project data and rolling-average engineered features are predictive of future cost variance and earned value.
    Feature engineering in Section III.A adds rolling averages of the target variables and assumes these carry forecasting signal for future performance.
  • ad hoc to paper Simulated weather and resource availability, generated under 'reasonable assumptions', faithfully represent the external factors that affect real project performance.
    Section III.A states that external data were unavailable and factors were simulated 'to demonstrate the potential impact' of external factors. All external-factor conclusions inherit this assumption.
  • standard math Standard ARIMA and LSTM assumptions apply to this project time series, such as stationarity for ARIMA and sufficient data for LSTM training.
    Section III.B applies standard implementations without diagnostic checks for stationarity, seasonality, or data volume.
invented entities (2)
  • Simulated weather patterns
    purpose: Serve as an external factor feature to predict cost variance and earned value.
    Section III.A: real weather data were unavailable, so a weather pattern variable was simulated. There is no real-world measurement to validate it, and the reported feature importance for weather is based only on this simulated variable.
  • Simulated resource availability
    purpose: Serve as an external factor feature to predict cost variance and earned value.
    Section III.A: resource availability was simulated because the original dataset lacked it. The conclusion that resource availability influences performance is an artifact of this simulated input.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enhancing Project Performance Forecasting using Machine Learning Techniques." pith.science (2026). https://pith.science/paper/5SRUKQUN

@misc{pith2026241117914,
  author       = {Pith},
  title        = {Pith review of: Enhancing Project Performance Forecasting using Machine Learning Techniques},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5SRUKQUN}},
  note         = {Machine review of arXiv:2411.17914}
}
read the original abstract

Accurate forecasting of project performance metrics is crucial for successfully managing and delivering urban road reconstruction projects. Traditional methods often rely on static baseline plans and fail to consider the dynamic nature of project progress and external factors. This research proposes a machine learning-based approach to forecast project performance metrics, such as cost variance and earned value, for each Work Breakdown Structure (WBS) category in an urban road reconstruction project. The proposed model utilizes time series forecasting techniques, including Autoregressive Integrated Moving Average (ARIMA) and Long Short-Term Memory (LSTM) networks, to predict future performance based on historical data and project progress. The model also incorporates external factors, such as weather patterns and resource availability, as features to enhance the accuracy of forecasts. By applying the predictive power of machine learning, the performance forecasting model enables proactive identification of potential deviations from the baseline plan, which allows project managers to take timely corrective actions. The research aims to validate the effectiveness of the proposed approach using a case study of an urban road reconstruction project, comparing the model's forecasts with actual project performance data. The findings of this research contribute to the advancement of project management practices in the construction industry, offering a data-driven solution for improving project performance monitoring and control.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Predicting the Impact of Scope Changes on Project Cost and Schedule Using Machine Learning Techniques

    cs.LG 2024-12 reject novelty 2.0 of 10

    Standard regression models predict simulated scope-change cost and schedule impacts on a small construction dataset, but no real-world validation is provided.

Reference graph

Works this paper leans on

16 extracted references · 16 canonical work pages · cited by 1 Pith paper

  1. [1]

    Kerzner, Project management metrics, KPIs, and dashboards: a guide to measuring and monitoring project performance

    H. Kerzner, Project management metrics, KPIs, and dashboards: a guide to measuring and monitoring project performance . John Wiley & Sons, 2022

    work page 2022

  2. [2]

    AI in project management: exploring theoretical models for decision -making and risk management,

    O. A. Odejide and T. E. Edunjobi, "AI in project management: exploring theoretical models for decision -making and risk management," Engineering Science & Technology Journal, vol. 5, no. 3, pp. 1072-1085, 2024

    work page 2024

  3. [3]

    Using Monte Carlo simulation to evaluate performance of forecasting models in project control,

    R. Ead, "Using Monte Carlo simulation to evaluate performance of forecasting models in project control," unpublished

    work page

  4. [4]

    Integrating machine learning and network analytics to model project cost, time and quality performance,

    S. Uddin, S. Ong, H. Lu, and P. Matous, "Integrating machine learning and network analytics to model project cost, time and quality performance," Production Planning & Control, pp. 1-15, 2023

    work page 2023

  5. [5]

    Presenting a hybrid scheme of machine learning combined with metaheuristic optimizers for predicting final cost and time of project,

    R. Bakhshi, S. F. Moradinia, R. Jani, and R. V. Poor, "Presenting a hybrid scheme of machine learning combined with metaheuristic optimizers for predicting final cost and time of project," KSCE Journal of Civil Engineering, vol. 26, no. 8, pp. 3188-3203, 2022

    work page 2022

  6. [6]

    Vanhoucke, Measuring time: Improving project performance using earned value management (Vol

    M. Vanhoucke, Measuring time: Improving project performance using earned value management (Vol. 136). Springer Science & Business Media, 2009

    work page 2009

  7. [7]

    Measuring the impact of scope changes on project plan using EVM,

    S. Tariq, N. Ahmad, M. U. Ashraf, A. M. Alghamdi, and A. S. Alfakeeh, "Measuring the impact of scope changes on project plan using EVM," IEEE Access, vol. 8, pp. 154589-154613, 2020

    work page 2020

  8. [8]

    The application of artificial intelligence in project management research: A review,

    J. Gil, J. Martinez Torres, and R. González -Crespo, "The application of artificial intelligence in project management research: A review," International Journal of Project Management, submitted for publication

    work page

Show all 16 references
  1. [9]

    Exploring the challenges and impacts of artificial intelligence implementation in project management: A systematic literature review,

    M. I. Hashfi and T. Raharjo, "Exploring the challenges and impacts of artificial intelligence implementation in project management: A systematic literature review," International Journal of Advanced Computer Science and Applications, vol. 14, no. 9, pp. 145-157, 2023

    2023

  2. [10]

    A review on deep sequential models for forecasting time series data,

    D. M. Ahmed, M. M. Hassan, and R. J. Mstafa, "A review on deep sequential models for forecasting time series data," Applied Computational Intelligence and Soft Computing , vol. 2022, no. 1, pp. 6596397, 2022

    2022

  3. [11]

    A comparison of ARIMA and LSTM in forecasting time series,

    S. Siami -Namini, N. Tavakoli, and A. S. Namin, "A comparison of ARIMA and LSTM in forecasting time series," in Proc. 17th IEEE Int. Conf. Machine Learning and Applications (ICMLA), Orlando, FL, USA, 2018, pp. 1394-1401

    2018

  4. [12]

    Modeling long-and short-term temporal patterns with deep neural networks,

    G. Lai, W. C. Chang, Y. Yang, and H. Liu, "Modeling long-and short-term temporal patterns with deep neural networks," in Proc. 41st Int. ACM SIGIR Conf. Research & Development in Information Retrieval , Ann Arbor, MI, USA, 2018, pp. 95-104

    2018

  5. [13]

    Day -ahead solar irradiance forecasting for microgrids using a long short -term memory recurrent neural network: A deep learning approach,

    M. Husein and I. Y. Chung, "Day -ahead solar irradiance forecasting for microgrids using a long short -term memory recurrent neural network: A deep learning approach," Energies, vol. 12, no. 10, pp. 1856, 2019

    2019

  6. [14]

    System dynamics approach for forecasting performance of construction projects,

    H. Leon, H. Osman, M. Georgy, and M. Elsaid, "System dynamics approach for forecasting performance of construction projects," Journal of Management in Engineering, vol. 34, no. 1, pp. 04017049, 2018

    2018

  7. [15]

    Work rate modeling of building construction projects using system dynamic to optimize project cost and time performance,

    F. Rachmawati, M. Mudjahidin, and E. Dewi Widowati, "Work rate modeling of building construction projects using system dynamic to optimize project cost and time performance," International Journal of Construction Management, vol. 24, no. 2, pp. 213-225, 2024

    2024

  8. [16]

    Developing a dataset of real projects for portfolio, program and project control management research,

    B. Thiele, M. Ryan, and A. Abbasi, "Developing a dataset of real projects for portfolio, program and project control management research," Data in Brief, vol. 34, pp. 106659, 2021. View publication stats

    2021

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.