Pith. sign in

REVIEW 3 major objections 6 minor 23 references

Deep learning for predicting hauling fleet production capacity under uncertainties in open pit mines using real and simulated data

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Pith's one-line read This paper claims that next-shift hauling-fleet payload in an open-pit mine can be forecast with a median absolute error of 8.4% when scheduled truck counts are added to an XGBoost model trained on rainfall, telemetry, and simulated…

desk verdict A plausible but unproven short-term payload forecast for one open-pit mine; the lookahead ambiguity in the rainfall feature and an abstract that overstates simulated data are the things to pin down before trusting the headline numbers. read the letter →

arxiv 2506.04296 v1 pith:5FN3L7RW submitted 2025-06-04 cs.LG

classification cs.LG
keywords open-pitmininghaulingfleetpayloadforecastingXGBoostLSTMSHAPanalysisrainfallsimulatedbreakdowndata
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that short-term hauling-fleet production capacity in an open-pit mine can be predicted one shift ahead from operational telemetry, rainfall records, and simulated equipment-availability scenarios. On a three-year dataset from a large tropical mine, the authors report that an XGBoost regressor achieves a median absolute error of 14.3% on next-shift payload, an LSTM reaches 15.1%, and adding the scheduled number of trucks for the next shift improves XGBoost to 8.4% with an R2 of 0.78. SHAP analysis identifies cumulative rainfall, recent payload history, and shift and crew identity as the dominant predictors, while the simulated breakdown scenarios contribute comparatively little. If the results hold in live operation, mine planners could anticipate rain-driven low-production shifts and adjust fleet assignments proactively.

What carries the argument

The load-bearing object is the engineered feature set that assembles three data families: empirical fleet telemetry (payload, cycle count, working trucks and shovels per shift), high-resolution ERA5-Land cumulative rainfall aggregated over the mine area, and simulated next-shift availability of trucks and shovels generated by regression models with Monte Carlo noise injection inspired by Monte Carlo Tree Search. These features are augmented with temporal lags—previous-shift payload, a four-shift rolling sum of payload, mean working trucks over four shifts, and six-shift cumulative rainfall—so that each prediction carries both immediate and aggregated history. Two architectures consume this feature set: XGBoost, a gradient-boosted tree regressor chosen for interpretability, and an LSTM, a gated recurrent network with an attention mechanism, chosen for temporal dependencies. SHAP values on the XGBoost model provide the claimed attribution of predictions to features, and the addition of a scheduled-truck count for the next shift is the modification that produces the best reported accuracy.

What would settle it

Compare the XGBoost model's median absolute error when 'Precipitation_next' is taken from a genuine weather forecast issued before the shift versus when it is taken from the observed rainfall record; if error rises materially with the forecast version, the headline 8.4% figure depends on future information. A simpler check is to delete 'Precipitation_next' entirely and see whether the remaining lagged rainfall features preserve the reported accuracy.

Watch

Extended reading notes

Core claim

The paper's central claim is that a hybrid dataset—real fleet-management telemetry, ERA5-Land rainfall, and regression-plus-Monte-Carlo simulations of next-shift truck and shovel availability—is sufficient to forecast total shift payload with practically useful accuracy. The authors show XGBoost and LSTM both track the broad payload trends over 379 test shifts, with XGBoost better at responding to sharp peaks and drops and LSTM producing smoother but lagged predictions. The decisive improvement comes from adding operational planning variables: when the scheduled number of working trucks for the target shift is included, XGBoost's median absolute error drops from 14.3% to 8.4% and its coefficient of determination reaches 0.78. The paper also claims, on the strength of SHAP values, that upcoming rainfall and cumulative rainfall over the last six shifts materially reduce predicted payload, and that simulated breakdown frequencies have only weak predictive impact.

Load-bearing premise

The reported accuracy rests on the assumption that the rainfall for the target shift is known when the forecast is made; if it is the observed rainfall from the future instead of a forecast, the model is being evaluated with information that would not be available to a planner.

Editorial extensions

If this is right

  • If the model is deployed, dispatchers can be warned one shift ahead of likely low-payload shifts caused by heavy rain, letting them reposition trucks or schedule maintenance.
  • Adding shift-planning variables such as the scheduled truck count is a cheap, high-impact way to stabilise forecasts; the paper reports it cuts XGBoost's errors exceeding 50% from 33 instances to 8.
  • XGBoost is the stronger candidate for operational use in this setting, both for accuracy and for SHAP-based interpretability; LSTM may serve when trend smoothing is desired.
  • The weak SHAP impact of simulated breakdown scenarios indicates the current simulation layer does not yet carry the predictive weight it was designed for, motivating the proposed integration of MTBF, MTTR, and crew data.
  • The framework can be embedded in dispatch decision-support systems to move from reactive to proactive fleet management in open-pit mines.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported accuracy depends on the assumption that the target-shift rainfall value is available at decision time; if 'Precipitation_next' is the observed historical rainfall rather than a genuine forecast, the headline errors likely understate real forecasting difficulty.
  • The paper's own finding that simulated breakdown variables have weak impact suggests the Monte Carlo simulation layer may be unnecessary for near-term accuracy; a simpler model using only telemetry, planned trucks, and rainfall could be compared as a cheaper baseline.
  • The same shift-level feature recipe—lagged production, weather, and planned resources—could transfer to other cyclical heavy industries such as ports or construction, though the paper does not claim this.
  • A direct testable extension is to retrain the XGBoost model with 'Precipitation_next' replaced by a genuine forecast issued before the shift and measure the accuracy gap.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper develops two models, XGBoost and LSTM, to forecast next-shift average payload (tons) for a hauling fleet in an open-pit mine. Inputs are shift-level FMS telemetry, ERA5-Land rainfall aggregates, temporal lag features, and simulated next-shift truck and shovel counts generated by regression models with injected residual noise. On a chronological 80/20 split, the paper reports XGBoost MedAE of 14.3% and LSTM MedAE of 15.1%; after adding a scheduled-truck input, XGBoost is reported to reach MedAE 8.4% with R²=0.78. SHAP analysis is used to attribute predictions, with payload history and rainfall reported as dominant predictors, while the text also states that simulated variables have weak impact.

Significance. If the evaluation is leakage-free, the paper provides a useful demonstration that moderate-accuracy next-shift payload forecasts are achievable for a specific open-pit operation using FMS telemetry and reanalysis rainfall data, with SHAP-based interpretability. The reported accuracy in the augmented XGBoost model is plausible and potentially relevant for short-term operational planning. However, the contribution is empirical and site-specific: no code or data are provided, no baselines are compared, and the headline claim about simulated breakdown data as a dominant predictor is contradicted by the paper's own SHAP results. These issues must be resolved before the central claims can be accepted.

major comments (3)
  1. [Materials and Methods, Weather data; Table 1] The feature 'Precipitation_next' is defined in Table 1 as 'Historical rainfall for the next shift' and sourced from ERA5-Land, which is a reanalysis product rather than a weather forecast. If this feature contains the observed rainfall for the target shift, then both models are trained and tested with information from the future, and the reported MedAE values (14.3%, 15.1%, and 8.4%) do not represent genuine forecasting performance. This is load-bearing because the SHAP analysis (Figures 7-8) and the text ('heavy forecasted rainfall has a significant negative impact') identify precipitation as one of the dominant predictors. Please state the provider and lead time of a true ex-ante rainfall forecast used at prediction time, or re-run the evaluation using only rainfall information available before the start of the target shift.
  2. [Abstract vs. Results, Figure 8] The abstract states that SHAP analyses identify 'simulated breakdown frequencies as dominant predictors,' but the Results section (Figure 8 and the accompanying paragraph) states that 'the three variables related to historical payload clearly dominate' and that 'simulated variables show a relatively weak impact.' These two statements cannot both be correct. Since the use of simulated breakdown scenarios is a central novelty claim, the authors should either provide evidence that simulated features materially improve predictions (e.g., an ablation study with and without them) or revise the abstract and framing accordingly.
  3. [Results, paragraph on error analysis] The improved XGBoost result (MedAE 8.4%, R²=0.78) is introduced only in the Results section, with no formal description of the experimental protocol: Table 1 does not list a 'number of trucks scheduled for the next shift' feature among the model inputs, and no ablation setup, hyperparameter retuning, or test-split definition is given for the augmented model. Without this information, the 8.4% result cannot be reproduced or compared with the 14.3% baseline. Please specify the feature, the exact data split, and the training and evaluation procedure for both configurations.
minor comments (6)
  1. [Materials and Methods, Weather data] 'January to Mars' should read 'January to March'; the phrase 'shift-by-shift of 10.5 hrs' is redundant and should be reworded.
  2. [Results, Figure 10] The caption says 'Kernel Density Estimate (KDE) of Absolute Errors' but the text describes percentage errors; please align the terminology and axes.
  3. [Results, Figure 11] 'As show in Figure 11' is a typo; the caption should also state explicitly that the first panel is XGBoost and the second is LSTM.
  4. [Materials and Methods, Simulated data] The simulation method is described as 'inspired by MCTS' but is actually a regression plus residual-noise Monte Carlo procedure; please clarify the connection to Monte Carlo Tree Search or remove the reference to MCTS.
  5. [Models' selection, LSTM] The LSTM description mentions an attention mechanism, but no details of its implementation or its contribution to performance are given; please either describe it or omit it.
  6. [Results] No comparison with simple baselines such as the persistence forecast (Payload_lag1) or a linear regression is provided; adding such baselines would help calibrate the reported MedAE values.

Circularity Check

1 steps flagged · score 6.0 of 10

Forecast evaluation leaks target-period rainfall through the 'Precipitation_next' feature; otherwise the empirical pipeline is not circular.

  1. other [Table 1, 'Precipitation_next' feature; Weather data section; Results/SHAP discussion]
    "Precipitation_next Meteorological Predictive Historical rainfall for the next shift ... we leverage the ERA5-Land hourly data set ... accessible via the Copernicus platform either through direct download or API queries ... heavy forecasted rainfall has a significant negative impact on productivity, more so than accumulated past rainfall."

    The feature 'Precipitation_next' is described as predictive but is sourced from ERA5-Land, a reanalysis/observational product, and defined as 'Historical rainfall for the next shift.' At the time a next-shift forecast would be made, the rainfall for that shift has not yet been observed. Feeding this value into the model means the reported MedAE values (14.3%, 15.1%, and 8.4% with R2=0.78) are not ex ante forecasts but results conditioned on the realized weather of the target shift. The central 'forecast' claim is therefore partially constructed from target-period information, making the evaluation circular in a temporal sense.

full rationale

This is an empirical supervised-learning study, so most of the claimed derivation chain is a standard train/test fit rather than a formal derivation. I found no load-bearing self-citation, no imported uniqueness theorem, and no ansatz smuggled in via citation. The simulated fleet-composition features are generated from regression models trained on FMS availability/utilization data, but the target (payload) is not used to generate them, and SHAP shows their impact is weak, so they do not create circularity. The one significant circularity-adjacent defect is temporal leakage: Table 1 defines 'Precipitation_next' as 'Historical rainfall for the next shift' from the ERA5-Land reanalysis product. Since ERA5-Land provides observed/estimated rainfall rather than an ex ante weather forecast, including this feature means the model is evaluated with target-period information. The paper itself calls this 'forecasted rainfall,' but the stated data source does not support that label. This invalidates the headline forecast errors as measures of genuine forecasting performance. I therefore flag this as the main circular step and assign a score of 6, while noting that this is primarily a validity/leakage defect rather than a definitional equivalence between the model output and its inputs.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on proprietary operational data and reanalysis weather data, hand-chosen hyperparameters and feature windows, and an unvalidated simulation procedure. No new physical entities are introduced.

free parameters (4)
  • XGBoost hyperparameters = n_estimators=1000, learning_rate=0.01, max_depth=3
    Chosen to balance complexity and generalization; no tuning procedure or search range is reported.
  • LSTM architecture hyperparameters = 64 units, dropout 0.2, Adam lr=0.001, look-back 10
    Hand-selected architecture choices; no sensitivity analysis is provided.
  • Feature lag windows = 4-shift rolling sums, 6-shift rainfall sum, 10-step look-back
    Chosen by the authors without evidence of systematic tuning; these windows shape the features.
  • Simulation residual noise distribution = unstated (sampled from model residuals)
    The simulation framework adds stochastic noise from residuals, but the exact distribution and regression models are not specified.
assumptions (4)
  • domain assumption ERA5-Land reanalysis precipitation approximates on-site rainfall.
    Weather features are taken from reanalysis grid points around the mine and treated as ground truth.
  • domain assumption Fleet management system data remain reliable after linear interpolation and forward-fill.
    Missing gaps are filled before training; the paper does not quantify interpolation error.
  • domain assumption The chronological split assumes the operational process is stationary enough for the trained models to generalize.
    Single 80/20 split, no accounting for regime changes over three years.
  • ad hoc to paper Simulated breakdown scenarios, generated from regression residuals, are plausible representations of future fleet availability.
    The paper presents residual injection as MCTS-inspired but provides no validation that simulated scenarios match real breakdown dynamics; the results show these features have weak predictive impact.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Deep learning for predicting hauling fleet production capacity under uncertainties in open pit mines using real and simulated data." pith.science (2026). https://pith.science/paper/5FN3L7RW

@misc{pith2026250604296,
  author       = {Pith},
  title        = {Pith review of: Deep learning for predicting hauling fleet production capacity under uncertainties in open pit mines using real and simulated data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5FN3L7RW}},
  note         = {Machine review of arXiv:2506.04296}
}
read the original abstract

Accurate short-term forecasting of hauling-fleet capacity is crucial in open-pit mining, where weather fluctuations, mechanical breakdowns, and variable crew availability introduce significant operational uncertainties. We propose a deep-learning framework that blends real-world operational records (high-resolution rainfall measurements, fleet performance telemetry) with synthetically generated mechanical-breakdown scenarios to enable the model to capture fluctuating high-impact failure events. We evaluate two architectures: an XGBoost regressor achieving a median absolute error (MedAE) of 14.3 per cent and a Long Short-Term Memory network with a MedAE of 15.1 per cent. Shapley Additive exPlanations (SHAP) value analyses identify cumulative rainfall, historical payload trends, and simulated breakdown frequencies as dominant predictors. Integration of simulated breakdown data and shift-planning features notably reduces prediction volatility. Future work will further integrate maintenance-scheduling indicators (Mean Time Between Failures, Mean Time to Repair), detailed human resource data (operator absenteeism, crew efficiency metrics), blast event scheduling, and other operational constraints to enhance forecast robustness and adaptability. This hybrid modelling approach offers a comprehensive decision-support tool for proactive, data-driven fleet management under dynamically uncertain conditions.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

23 extracted references · 17 canonical work pages

  1. [1]

    APCOM 2025 | Perth, Australia | 10–13 August 2025 1 Deep learning for predicting hauling fleet production capacity under uncertainties in open pit mines using real and simulated data N Guerin1, M Nakhla2, A Dehoux3 and J L Loyer4

  2. [2]

    Designed to remember long-term patterns, LSTMs are ideal for mining fleet data, where cyclic production schedules are common

    enables effective retention and selective forgetting of information over long sequences. Designed to remember long-term patterns, LSTMs are ideal for mining fleet data, where cyclic production schedules are common. By processing multivariate time-series sequences, the LSTM architecture learns to forecast key metrics, effectively modelling the dynamic evol...

  3. [4]

    Head of Data and AI, Eramet, Paris 75015, France. Email: jl.loyer@eramet.com ABSTRACT Accurate short-term forecasting of hauling-fleet capacity is crucial in open-pit mining, where weather fluctuations, mechanical breakdowns, and variable crew availability introduce significant operational uncertainties. We propose a deep-learning framework that blends re...

  4. [5]

    FIG 4 – Schematic of hauling operation in surface mines from Soofastaei et al (2015). Finally, the shift and crew variables categorise the operational periods, revealing that certain shifts experienced slightly faster cycle times, likely a result of differences in crew performance and working conditions. Collectively, these metrics, quantified through the...

  5. [7]

    FIG 7 – SHAP analysis: ‘upcoming rainfall’ feature impact on model predictions

    This granular insight not only validates our feature selection but also underscores the practical importance of including weather data in operational forecasts. FIG 7 – SHAP analysis: ‘upcoming rainfall’ feature impact on model predictions. To better understand the impact of each input feature on the model’s predictions, a sensitivity analysis was conduct...

  6. [10]

    XGBoost: Achieved a MedAE of 14.3 per cent

    12 APCOM 2025 | Perth, Australia | 10–13 August 2025 FIG 10 – Kernel Density Estimate (KDE) of Absolute Errors for XGBoost versus LSTM. XGBoost: Achieved a MedAE of 14.3 per cent. This model benefits from its capacity to model non-linear relationships and its enhanced interpretability via SHAP values, which clearly indicate the strong influence of rainfal...

  7. [12]

    This analysis underscores the importance of integrating operational factors into predictive modelling

    FIG 12 – Model calibration plot (XGBoost and LSTM). This analysis underscores the importance of integrating operational factors into predictive modelling. It also opens the door for future enhancements, such as incorporating maintenance scheduling and reliability indicators like Mean Time Between Failures (MTBF) and Mean Time to Repair (MTTR), which could...

  8. [13]

    APCOM 2025 | Perth, Australia | 10–13 August 2025 15 Carvalho, M, Sampaio, P and Rebentisch, E, Carvalho, J

    A framework to simplify the management of throughput and constraints, Southern African Institute of Mining and Metallurgy, Johannesburg. APCOM 2025 | Perth, Australia | 10–13 August 2025 15 Carvalho, M, Sampaio, P and Rebentisch, E, Carvalho, J. Á., & Saraiva, P. (2019). Operational excellence, organisational culture and agility: the missing link? Total Q...

Show all 23 references
  1. [15]

    https://doi.org/10.1080/17480930.2022.2142425 García, S, Luengo, J and Herrera, F,

    Prediction of truck productivity at mine sites using tree-based ensemble models combined with Gaussian mixture modelling, International Journal of Mining, Reclamation and Environment, 37(1):66–86. https://doi.org/10.1080/17480930.2022.2142425 García, S, Luengo, J and Herrera, F,

  2. [18]

    https://doi.org/10.1016/j.mineng.2023.108565 Soofastaei, A, Aminossadati, S, Kizil, M S and Knights, P,

    Integrated and intelligent remote operation centres (I2ROCs): Assessing the human–machine requirements for 21st century mining operations, Minerals Engineering, 207:108565. https://doi.org/10.1016/j.mineng.2023.108565 Soofastaei, A, Aminossadati, S, Kizil, M S and Knights, P,

  3. [21]

    https://doi.org/10.1007/s10462-020-09838-1 Wang, Q, Zhang, R, Lv, S and Wang, Y,

    A review on the long short-term memory model, Artificial Intelligence Review, 53:5929–5955. https://doi.org/10.1007/s10462-020-09838-1 Wang, Q, Zhang, R, Lv, S and Wang, Y,

  4. [22]

    https://doi.org/10.1016/j.seta.2020.100977 Wang, W, Chakraborty, G and Chakraborty, B,

    Open pit mine truck fuel consumption pattern and application based on multi-dimensional features and XGBoost, Sustainable Energy Technologies and Assessments, 43:100977. https://doi.org/10.1016/j.seta.2020.100977 Wang, W, Chakraborty, G and Chakraborty, B,

  5. [23]

    https://doi.org/10.3390/app11010202 16 APCOM 2025 | Perth, Australia | 10–13 August 2025

    Predicting the Risk of Chronic Kidney Disease (CKD) Using Machine Learning Algorithm, Applied Science, 11:202. https://doi.org/10.3390/app11010202 16 APCOM 2025 | Perth, Australia | 10–13 August 2025

  6. [2010]

    https://doi.org/10.1108/17542731011085325 Baek, J and Choi, Y,

    Meta-management of integration of management systems, The TQM Journal, 22(6):599–613. https://doi.org/10.1108/17542731011085325 Baek, J and Choi, Y,

  7. [2012]

    https://doi.org/10.1109/TCIAIG.2012.2186810 Cambitsis, A,

    A survey of Monte Carlo tree search methods, IEEE Transactions on Computational Intelligence and AI in Games, 4(1):1–43. https://doi.org/10.1109/TCIAIG.2012.2186810 Cambitsis, A,

  8. [2015]

    Tlhatlhetji, M and Kolapo, P,

    Simulation of payload variance effects on truck bunching to minimise energy consumption and greenhouse gas emissions, in Proceedings of the Coal Operators’ Conference 2015 (eds: N Aziz and B Kininmonth), pp 337–346 (University of Wollongong – Mining Engineering, the Australasi...

  9. [2016]

    https://doi.org/10.1145/2939672.2939785 Fan, C, Zhang, N, Jiang, B and Liu, W V,

    XGBoost: A scalable tree boosting system, in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, pp 785–794. https://doi.org/10.1145/2939672.2939785 Fan, C, Zhang, N, Jiang, B and Liu, W V,

  10. [2019]

    https://doi.org/10.1007/s11053-018-9396-1 Hochreiter, S and Schmidhuber, J,

    Evaluation of impact of potential extreme rainfall events on mining in Peru, Natural Resources Research, 28:393–408. https://doi.org/10.1007/s11053-018-9396-1 Hochreiter, S and Schmidhuber, J,

  11. [2020]

    https://doi.org/10.1016/j.resourpol.2020.101569 Shimaponda-Nawa, M and Nwaila, G T,

    Innovation in the mining industry: Technological trends and a case study of the challenges of disruptive innovation, Resources Policy, 65:101569. https://doi.org/10.1016/j.resourpol.2020.101569 Shimaponda-Nawa, M and Nwaila, G T,

  12. [2021]

    Investigating the effects of rainy season on open cast mining operation: The case of Wescoal Khanyisa Colliery, Research Square Preprint, https://doi.org/10.21203/rs.3.rs-870740/v1 Van Houdt, G, Mosquera, C and Napoles, G,

  13. [2022]

    Gonzalez et al (2019) evaluated the effects of extreme rainfall events on open-pit mines in Peru, demonstrating marked operational delays during heavy rains

    Incorporating cumulative precipitation measures into our deep learning model is crucial, as heavy rainfall episodes directly reduce operational efficiency by deteriorating haul road conditions, increasing cycle times, and occasionally forcing temporary equipment downtime due t...

  14. [2023]

    https://doi.org/10.23919/CCC58697.2023.10240705 Asif, M, Bessant, J and Francis, D,

    Prediction method of truck travel time in open pit mines based on LSTM model, in Proceedings of the 42nd Chinese Control Conference (CCC), pp 8651–8656. https://doi.org/10.23919/CCC58697.2023.10240705 Asif, M, Bessant, J and Francis, D,

  15. [2024]

    The region displays a clear seasonality, with a wet season spanning January to Mars, during which extreme events such as tropical depressions contribute to intense rainfall peaks

    Annual totals ranged from over 2500 mm in 2021—an exceptionally wet year marked by cyclonic activity—to just 1485 mm in 2024, reflecting a 6 per cent deficit relative to the 1991–2020 baseline. The region displays a clear seasonality, with a wet season spanning January to Mars...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.