REVIEW 3 major objections 7 minor 31 references
Machine Learning-Based Self-Localization Using Internal Sensors for Automating Bulldozers
T0 review · 3 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read This paper claims that a bulldozer can keep itself localized with sub-meter accuracy using only its internal sensors, by learning to recognize track slip from hydraulic and engine data and feeding the corrected velocity into an extended…
desk verdict ML-based velocity estimation from internal sensors gives sub-meter bulldozer localization in recorded scenarios; the main uncertainty is whether it generalizes to unseen sites, since the test set is tiny and not held out. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is a two-stage estimator. Stage one is a machine-learning velocity model, an LSTM with a 40-step sliding window (also tested with an MLP and XGBoost), that maps standardized internal-sensor features to the bulldozer's local x/y/z velocity. Stage two is an extended Kalman filter whose prediction step integrates that velocity with IMU angular velocity to propagate position and orientation, and whose update step corrects roll and pitch from the direction of gravity in the IMU acceleration. The load-bearing detail is that hydraulic pump pressure and blade-position readings change with the static-to-dynamic friction transition that marks the onset of track slip, which is what lets the model emit a corrected velocity instead of the raw track rotation.
What would settle it
Take the trained LSTM-plus-EKF system to a new site or a second bulldozer of the same model, run fresh excavation and slope episodes that have no overlap with the 442 training sequences, and compute the average displacement error against a survey-grade reference; if the error climbs into the multi-meter range when slip is frequent, the central claim is refuted. A cheaper, immediate check is to stratify the ten existing test episodes by soil type and slip fraction and verify that the 0.50 m average is not carried by a few benign scenarios.
Extended reading notes
Core claim
On the paper's own terms, the discovery is that track slip, which makes crawler odometry read high velocities while the machine barely moves, is predictable from the machine's hydraulic and engine state, so a supervised model can output a slip-corrected local velocity. The authors demonstrate that the learned velocity, used as the control input of an EKF with IMU roll/pitch updates, keeps the estimated global trajectory close to the RTK-GNSS ground truth even in the slip intervals where the kinematics-based EKF's error grows. The average displacement error across all ten scenarios is 0.50 m for LSTM with EKF, and the method suppresses final-position error in every scenario except grading, where the conventional EKF is slightly better.
Load-bearing premise
The load-bearing premise is that a velocity model trained on recorded runs, using RTK-GNSS-derived positions as ground truth, will keep estimating correctly on new ground, soil, loads, and slip conditions it never saw; if that generalization fails, the reported error suppression is limited to the recorded episodes rather than to bulldozer automation in general.
Editorial extensions
If this is right
- If the result transfers, bulldozer automation can continue during RTK-GNSS outages under bridges, in open-pit valley floors, or during ionospheric disturbances, without adding LiDAR or camera hardware.
- The method's average displacement error of 0.50 m across ten scenarios is roughly seven times better than a kinematics-based EKF and sixteen times better than crawler odometry, so slip compensation is the dominant source of gain.
- Adding bulldozer-specific sensors, blade position and hydraulic pump pressure, lowers mean ADE from 0.73 m to 0.50 m, meaning hydraulic signals carry useful slip information for localization, not just machine control.
- The LSTM variant runs at about 0.0061 seconds per 100 Hz frame on an embedded GPU, faster than the 0.01 s frame interval, so the accuracy gain does not cost real-time operation.
- Because the velocity estimate is plug-in, the same EKF shell could accept corrections from external sensors such as LiDAR or cameras when they are available.
Reading between the lines
- The report leaves open whether the learned slip signatures transfer across machines or sites: all 442 training episodes come from one bulldozer and five soil fields, so the 0.50 m figure is a same-machine, same-sensor result until cross-condition data appear.
- A natural extension the authors do not test is making the EKF's process noise state-dependent, so the filter inflates uncertainty when the learned model detects slip, rather than using a fixed diagonal Q.
- The hydraulic-pressure feature probably generalizes to other hydrostatically driven construction machines, so the same velocity-estimation-plus-EKF recipe could be retrained for wheel loaders or excavators, whose pump pressure also responds to tractive load and slip.
- Stratifying by soil type and slip fraction on the released dataset would sharpen the claim: currently the scenario-level averages mix soil conditions, leaving open which condition actually drives the improvement.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a GNSS-free self-localization pipeline for bulldozers: a machine-learning model (MLP, LSTM, or XGBoost) maps windows of internal-sensor data (IMU, crawler encoders, engine and hydraulic/blade sensors) to local three-axis velocity, and an extended Kalman filter integrates these velocity estimates with IMU angular rates and accelerometer-derived roll/pitch. The authors collected 462 episodes from a Komatsu D37PXI-24 across five soil areas and ten driving scenarios, trained on 442 sequences, and tested on one sequence per scenario. Table 3 reports an average ADE of 0.50 m for LSTM+EKF versus 3.47 m for a kinematics-based EKF and 8.10 m for crawler odometry, and Table 5 attributes part of the gain to bulldozer-specific sensors. Because RTK-GNSS is used only for supervision and evaluation, not as an inference input, the evaluation is not circular; the open question is whether the reported accuracy transfers to deployments outside the recorded conditions.
Significance. If the reported accuracy holds under deployment conditions, this is a practically significant step toward GNSS-denied localization for heavy equipment: sub-meter average ADE across slip-heavy scenarios using only internal sensors, real-time throughput on a Jetson AGX Orin, and a large multi-scenario dataset from a commercial bulldozer. Strengths of the paper include a realistic dataset spanning five soil areas and ten maneuvers, a clear EKF formulation, an ablation isolating the contribution of bulldozer-specific sensors, and explicit real-time timing measurements. The principal caveat is that the empirical support for generalization is narrow: the test set is one hand-selected trajectory per scenario with no held-out soil, field, or operator split, so the 0.50 m average should be read as an in-distribution demonstration rather than established deployment accuracy.
major comments (3)
- [Section 5.1, Table 3] The test set consists of 10 sequences, one per scenario, and the selection criterion is unspecified; with 442 training plus 10 validation plus 10 test sequences equaling the 462 recorded episodes from only five soil areas, the split does not demonstrate transfer to unseen fields, soil types, or operators. Because the motivating use case is GNSS loss at deployment, a leave-one-field-out or leave-one-soil-type-out evaluation, together with multiple test episodes per scenario and an explicit selection rule, is needed before the 0.50 m average ADE can support the generalization claim.
- [Table 3] The reported plus-or-minus values are across five independent training runs, not across test trajectories; each scenario contributes exactly one trajectory, so the average of 0.50 +/- 0.05 m does not quantify trajectory-level variability. Per-trajectory ADE values, multiple test sequences per scenario, and confidence intervals are required; without them, the scenario-level rankings (for example, XGBoost at 0.21 m versus LSTM at 0.27 m on crossing slope) cannot be distinguished from chance.
- [Sections 2 and 5.3] The related work surveys learning-based odometry baselines such as Brossard and Bonnabel (2019), Onyekpe et al. (2021), and Choi et al. (2024), but the localization evaluation compares only crawler odometry and a kinematics-based EKF. Since those learned methods are the closest prior art for internal-sensor odometry, the absence of any such comparison leaves open whether the proposed pipeline improves over existing learning-based odometry or only over kinematic models; adding at least one recent learned baseline, or a clear argument why it cannot be applied to bulldozer HST/crawler signals, would substantiate the contribution claim.
minor comments (7)
- [Tables 3 and 5] Table 3 states that averages and standard deviations are calculated from five independent trials, while Table 5 states three independent trials; the identical reported mean and standard deviation (0.50 +/- 0.05) for the same LSTM+EKF configuration with IC+Ve+Bu inputs in both tables needs clarification.
- [Section 5.1] With each sequence containing between 3,000 and 20,000 time steps, 442 training sequences cannot produce about 47,000,000 samples; if this count refers to individual sensor values rather than time steps, the manuscript should state that explicitly.
- [Figure 8 caption] The caption refers to 'slalom while excavating,' but the corresponding scenario in Figure 6 and in the text is 'slalom while carrying dirt'; this should be corrected.
- [Tables 2 and 3] MLP and LSTM rows report mean +/- standard deviation, but XGBoost rows report only point values; the same uncertainty reporting should be provided for XGBoost.
- [Section 5.1] The statement that XGBoost was trained using default settings provided by scikit-learn 1.3.2 is imprecise, since XGBoost is an external library; the exact package and version should be specified.
- [Section 3.2.2, Eq. (10)] The use of tan^{-1} without atan2 leaves quadrant and sign conventions ambiguous for the roll and pitch observations; please specify the intended atan2 formulation.
- [Section 5.3] The initialization of the EKF (initial pose, yaw, and initial covariance P_0) is not described; the manuscript should state whether all methods start from the same ground-truth initial pose.
Circularity Check
No significant circularity: the learned velocity model is supervised against RTK-GNSS labels, and GNSS is not an input at inference, so the localization results are not forced by construction.
full rationale
The paper's derivation chain is a standard supervised learned-odometry pipeline. A machine learning model (MLP, LSTM, or XGBoost) maps internal sensor features to local velocities; those velocities feed an EKF for global localization. The ground-truth labels and evaluation positions come from RTK-GNSS plus IMU (Table 1 and Section 4), but RTK-GNSS is explicitly not an input to the model at inference: 'The pose of the vehicle is estimated using GNSS and IMU when the GNSS signals are stably received. This estimated pose data is used as the target variable and is not input into the proposed model.' The prediction target (local velocity) is a different quantity from the downstream evaluation metric (global position ADE, Eq. 16), and the EKF integration is a physically specified state-transition model (Eq. 7), not a fitted surrogate for ADE. The comparisons against crawler odometry and a kinematics-based EKF (Section 5.3) are independent baselines that share the same EKF structure for the kinematics-based EKF but replace only the velocity input, so the LSTM/MLP/XGBoost improvements are attributable to the learned velocity estimates. There are no equations in which an output is defined in terms of the quantity it is claimed to predict, no fitted parameter is renamed as a prediction, and no load-bearing uniqueness theorem is imported from prior work by the same authors. The cited prior work (e.g., Brossard and Bonnabel 2019, Choi et al. 2024) is external and used for context, not to justify the proposed method's correctness. The main legitimate weakness is generalization risk: the test set is 10 hand-picked sequences with no held-out field or soil split, and the ADE standard deviations in Table 3 are across training runs rather than trajectories. That is an empirical validity concern, not circularity. Therefore the circularity score is 0.
Assumptions & free parameters
free parameters (4)
- LSTM window size N =
40
- MLP and LSTM architecture sizes =
4 hidden layers x 256 units (MLP), 256 LSTM units
- Process noise covariance Q coefficients =
diag(0.1*(dt)^2 x3, 0.01*(dt)^2 x3)
- Measurement noise covariance R =
diag(0.01, 0.01)
assumptions (4)
- domain assumption RTK-GNSS+IMU pose estimates used as ground truth are accurate enough to serve as training labels and evaluation reference.
- domain assumption The learned mapping from internal sensors to local velocity generalizes from the 442 training episodes to the 10 test sequences and to unseen GNSS-free deployments.
- domain assumption Roll and pitch can be inferred from the accelerometer under the rest or constant-velocity assumption.
- standard math Standard EKF and rotation matrix formulas are correct and applicable.
Cite this review
Pith. "Pith review of Machine Learning-Based Self-Localization Using Internal Sensors for Automating Bulldozers." pith.science (2026). https://pith.science/paper/7BABRFUG
@misc{pith2026250607271,
author = {Pith},
title = {Pith review of: Machine Learning-Based Self-Localization Using Internal Sensors for Automating Bulldozers},
year = {2026},
howpublished = {\url{https://pith.science/paper/7BABRFUG}},
note = {Machine review of arXiv:2506.07271}
}
read the original abstract
Self-localization is an important technology for automating bulldozers. Conventional bulldozer self-localization systems rely on RTK-GNSS (Real Time Kinematic-Global Navigation Satellite Systems). However, RTK-GNSS signals are sometimes lost in certain mining conditions. Therefore, self-localization methods that do not depend on RTK-GNSS are required. In this paper, we propose a machine learning-based self-localization method for bulldozers. The proposed method consists of two steps: estimating local velocities using a machine learning model from internal sensors, and incorporating these estimates into an Extended Kalman Filter (EKF) for global localization. We also created a novel dataset for bulldozer odometry and conducted experiments across various driving scenarios, including slalom, excavation, and driving on slopes. The result demonstrated that the proposed self-localization method suppressed the accumulation of position errors compared to kinematics-based methods, especially when slip occurred. Furthermore, this study showed that bulldozer-specific sensors, such as blade position sensors and hydraulic pressure sensors, contributed to improving self-localization accuracy.
Reference graph
Works this paper leans on
-
[1]
bchapter Brossard , M. , Bonnabel , S. : Learning wheel odometry and IMU errors for localization . In: 2019 International Conference on Robotics and Automation ( ICRA ) , pp. 291 -- 297 . IEEE , Montreal, Canada ( 2019 ) bchapter
work page 2019
-
[2]
: Quadrotor dynamics and control rev 0.1
botherref Beard , R. : Quadrotor dynamics and control rev 0.1. Technical report, Brigham Young University (2008). http://hdl.lib.byu.edu/1877/624 botherref
work page 2008
-
[3]
botherref Borenstein , J. , Feng , L. : Gyrodometry: A new method for combining data from gyros and odometry in mobile robots. Proceedings of the 1996 IEEE International Conference on Robotics and Automation, 423--428 (1996) botherref
work page 1996
-
[4]
, Liqiang Feng : Measurement and correction of systematic odometry errors in mobile robots
barticle Borenstein , J. , Liqiang Feng : Measurement and correction of systematic odometry errors in mobile robots . IEEE Transactions on Robotics and Automation 12 ( 6 ), 869 -- 880 ( 1996 ) 10.1109/70.544770 barticle
-
[5]
barticle Belhajem , I. , Maissa , Y.B. , Tamtaoui , A. : Improving Vehicle Localization in a Smart City with Low Cost Sensor Networks and Support Vector Machines . Mobile Networks and Applications 23 ( 4 ), 854 -- 863 ( 2018 ) 10.1007/s11036-017-0879-9 barticle
-
[6]
bchapter Chen , T. , Guestrin , C. : XGBoost : A Scalable Tree Boosting System . In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery And Data Mining , pp. 785 -- 794 ( 2016 ). 10.1145/2939672.2939785 bchapter
arXiv 2016
-
[7]
barticle Chiang , K.-W. , Noureldin , A. , El-Sheimy , N. : Multisensor integration using neuron computing for land-vehicle navigation . GPS Solutions 6 ( 4 ), 209 -- 218 ( 2003 ) 10.1007/s10291-002-0024-4 barticle
-
[8]
barticle Choi , S. , Shon , H. , Huh , K. : Interpretable Vehicle Speed Estimation Based on Dual Attention Network for 4WD Off - Road Vehicles . IEEE Transactions on Intelligent Vehicles 9 ( 1 ), 151 -- 164 ( 2024 ) 10.1109/TIV.2023.3323283 barticle
arXiv 2024
Show all 31 references
-
[9]
, Maher , A
barticle El Sabbagh , M.S. , Maher , A. , Abozied , M.A.H. , Kamel , A.M. : Promoting navigation system efficiency during GPS outage via cascaded neural networks: A novel AI based approach . Mechatronics 94 , 103026 ( 2023 ) 10.1016/j.mechatronics.2023.103026 barticle
2023
-
[10]
, Ishibashi , E
botherref Fukasawa , T. , Ishibashi , E. : Small-sized crawler dozers ``D31/37/39-21'' with electronically controlled hst. Komatsu technical report 50(153) (2004) botherref
2004
-
[11]
, Jiang , J
barticle Fang , W. , Jiang , J. , Lu , S. , Gong , Y. , Tao , Y. , Tang , Y. , Yan , P. , Luo , H. , Liu , J. : A LSTM Algorithm Estimating Pseudo Measurements for Aiding INS during GNSS Signal Outages . Remote Sensing 12 ( 2 ), 256 ( 2020 ) 10.3390/rs12020256 barticle
2020 doi
-
[12]
, Syed , Z
bchapter Goodall , C. , Syed , Z. , El-Sheimy , N. : Improving INS / GPS Navigation Accuracy through Compensation of Kalman Filter Errors . In: IEEE Vehicular Technology Conference , pp. 1 -- 5 ( 2006 ). https://ieeexplore.ieee.org/abstract/document/4109843/ bchapter
2006
-
[13]
, Guivant , J
barticle Hirayama , M. , Guivant , J. , Katupitiya , J. , Whitty , M. : Path planning for autonomous bulldozers . Mechatronics 58 , 20 -- 38 ( 2019 ) barticle
2019
-
[14]
, Nunotani , S
botherref Hayashi , S. , Nunotani , S. , Mukaino , M. : Hydraulic Systems and Components for Medium -sized Bulldozer 2008(7-1), 65--68 (2008). Proceedings of the JFPS International Symposium on Fluid Power botherref
2008
-
[15]
, Schmidhuber , J
barticle Hochreiter , S. , Schmidhuber , J. : Long Short - Term Memory . Neural Computation 9 ( 8 ), 1735 -- 1780 ( 1997 ) 10.1162/neco.1997.9.8.1735 . Conference Name: Neural Computation barticle
1997 doi
-
[16]
: A new approach to linear filtering and prediction problems
botherref Kalman , R.E. : A new approach to linear filtering and prediction problems. Journal of Basic Engineering, 35--45 (1960) botherref
1960
-
[17]
, Ba , J
bchapter Kingma , D.P. , Ba , J. : Adam: A method for stochastic optimization . In: 3rd International Conference on Learning Representations, ICLR 2015 ( 2015 ) bchapter
2015
-
[18]
: DNN - Based Slip Ratio Estimator for Lugged - Wheel Robot Localization in Rough Deformable Terrains
botherref Kim , C.-h. : DNN - Based Slip Ratio Estimator for Lugged - Wheel Robot Localization in Rough Deformable Terrains . IEEE Access (2023). Publisher: IEEE botherref
2023
-
[19]
, Nakano , Y
barticle Kamikawa , S. , Nakano , Y. : Bulldozer D71-24 . Komatsu technical report 66 ( 173 ), 44 -- 50 ( 2020 ) barticle
2020
-
[20]
, Guo , G
barticle Liu , J. , Guo , G. : Vehicle Localization During GPS Outages With Extended Kalman Filter and Deep Learning . IEEE Transactions on Instrumentation and Measurement 70 , 1 -- 10 ( 2021 ) 10.1109/TIM.2021.3097401 barticle
2021
-
[21]
, Kolu , A
botherref Mononen , T. , Kolu , A. , Mattila , J. : Semi-autonomous bulldozer blade control using real-time terrain mapping. Proceedings of 2022 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), 775--782 (2022) botherref
2022
-
[22]
, Hinton , G.E
bchapter Nair , V. , Hinton , G.E. : Rectified Linear Units Improve Restricted Boltzmann Machines . In: Proceedings of the 27th International Conference on Machine Learning (ICML-10) , pp. 807 -- 814 ( 2010 ) bchapter
2010
-
[23]
, Palade , V
barticle Onyekpe , U. , Palade , V. , Herath , A. , Kanarachos , S. , Fitzpatrick , M.E. : WhONet : Wheel Odometry neural Network for vehicular localisation in GNSS -deprived environments . Engineering Applications of Artificial Intelligence 105 , 104421 ( 2021 ) 10.1016/j.eng...
2021
-
[24]
, Varoquaux , G
barticle Pedregosa , F. , Varoquaux , G. , Gramfort , A. , Michel , V. , Thirion , B. , Grisel , O. , Blondel , M. , Prettenhofer , P. , Weiss , R. , Dubourg , V. , Vanderplas , J. , Passos , A. , Cournapeau , D. , Brucher , M. , Perrot , M. , Duchesnay , E. : Scikit-learn: Ma...
2011
-
[25]
: Kalman and extended K alman filters: Concept, derivation and properties
botherref Ribeiro , M.I. : Kalman and extended K alman filters: Concept, derivation and properties. Institute for Systems and Robotics 43 (2004) botherref
2004
-
[26]
, Hu , M.Y
barticle Shanker , M. , Hu , M.Y. , Hung , M.S. : Effect of data standardization on neural network training . Omega 24 ( 4 ), 385 -- 397 ( 1996 ) 10.1016/0305-0483(96)00010-2 barticle
1996 doi
-
[27]
, Weston , J.L
bbook Titterton , D. , Weston , J.L. , Weston , J. : Strapdown Inertial Navigation Technology . IET , Stevenage ( 2004 ) bbook
2004
-
[28]
, Xia , Z.C
botherref Taheri , H. , Xia , Z.C. : SLAM ; definition and evolution. Engineering Applications of Artificial Intelligence 97 (2021) botherref
2021
-
[29]
, Joly , C
bchapter Valente , M. , Joly , C. , La Fortelle , A. : An LSTM Network for Real - Time Odometry Estimation . In: 2019 IEEE Intelligent Vehicles Symposium ( IV ) , pp. 1434 -- 1440 ( 2019 ). 10.1109/IVS.2019.8814133 . ISSN: 2642-7214 bchapter
2019
-
[30]
, Nagatani , K
barticle Yamauchi , G. , Nagatani , K. , Hashimoto , T. , Fujino , K. : Slip-compensated odometry for tracked vehicle on loose and weak slope . ROBOMECH Journal 4 ( 1 ), 27 ( 2017 ) 10.1186/s40648-017-0095-1 barticle
2017 doi
-
[31]
write newline
" write newline "" before.all 'output.state := FUNCTION string.to.integer 't := t text.length 'k := #1 'char.num := t char.num #1 substring 's := s is.num s "." = or char.num k = not and char.num #1 + 'char.num := while char.num #1 - 'char.num := t #1 char.num substring FUNCTI...
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.