REVIEW 3 major objections 5 minor 27 references
A real-time RGB-D pipeline can generate feasible rock-breaking poses for autonomous impact hammers, with a 72% stable-contact success rate in scaled real-world trials.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 09:29 UTC pith:WLEOHSOC
load-bearing objection Solid integration paper with a real hardware test, but the headline 72% is a stable-contact rate, not a rock-fracture rate, and the conclusion overstates it. the 3 major comments →
A real-time RGB-D perception pipeline for autonomous impact hammers in mining: self-filtering, rock segmentation and rock-breaking poses generation
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that surface-normal analysis over segmented rock point clouds, combined with the kinematic and operational constraints of a four-degree-of-freedom impact hammer, produces rock-breaking poses that are operationally feasible. Rather than placing one target at each rock's centroid, the method clusters points with similar surface orientations, defines a proto pose at the point closest to each cluster's centroid with orientation equal to the mean normal, and then projects that normal onto the plane defined by the target position and the hammer's rotation axis to obtain a reachable pitch angle. Thresholds discard poses that would strike too obliquely. In real-world pressing tr
What carries the argument
The load-bearing mechanism is the surface-normal-based pose generator. For each segmented rock point cloud, it computes local surface normals, clusters points with similar orientations via density-based clustering, forms a proto pose at the point closest to each cluster's centroid with orientation equal to the mean normal, projects that normal onto the t-plane defined by the target position and the hammer's rotation center to fix the pitch angle, and then applies soft and hard pitch-angle filters to keep only near-vertical, reachable strikes. This encodes the operational rule 'strike flat surfaces nearly perpendicular to the grizzly' directly into the pose geometry, which the paper credits f
Load-bearing premise
The key assumption is that a laboratory-scale scene—a mini-excavator fixed beside a 1.5-meter steel grate with two ceiling cameras—reproduces the conditions of a real underground ore-pass, including dust, lighting, full-size grizzlies, and machine dynamics; the 10 Hz throughput and 72% success rate were measured in that scaled setup.
What would settle it
A full-scale field trial at an actual ore pass, running the same pipeline on embedded hardware, in which the proportion of target poses that achieve stable chisel contact is recorded over a few hundred attempts; if the success rate falls well below 72% or the 10 Hz update cannot be sustained in dust and variable lighting, the paper's suitability claim is falsified.
If this is right
- The pipeline can be paired with a reaching controller and high-level state machines to close the loop: the hammer returns to a home configuration, selects the highest-priority pose, presses, and repeats, as demonstrated in the real-world validation.
- At roughly 10 Hz with about 675 ms end-to-end latency, the system provides a continuous stream of target poses that is compatible with responsive closed-loop control on embedded hardware, without offline processing.
- The robot-free 3D workspace representation, produced alongside the poses and with occlusions filled by the background model, can be consumed by downstream modules such as collision avoidance and motion planning.
- Because poses are derived from local surface geometry rather than rock instance identity, imperfect segmentation does not block pose generation; it mainly perturbs the prioritization of poses, for example through rock-volume estimates.
- The method is intentionally conservative: rocks with highly inclined surfaces may receive no pose at all, a design choice the paper argues is safer than proposing a slippage-prone target.
- This is a field-tested extension: prior perception systems for rock breaking either used simple centroid targets or did not jointly enforce hammer kinematics and surface geometry, and this paper's contribution is precisely that joint enforcement running in real time.
Where Pith is reading between the lines
- A stronger end-to-end test would fire the hammer rather than press: stable contact under sustained pressure does not guarantee effective energy transfer during an actual impact, so an impact trial could change the 72% figure.
- The robot-free 3D workspace representation is a natural substrate for non-prehensile manipulation planning—repositioning unstable boulders by pushing—which the paper mentions only as future work.
- The score weights and angular thresholds are hand-tuned for this setup; an adaptive or learned prioritization that maps geometric features to breakage outcomes could improve generalization across mines and rock types.
- Because the pipeline assumes a quasi-static scene and ceiling-mounted cameras, transferring to a moving or differently placed sensor would require the optional point-cloud-based self-filter, whose latency was not measured separately in the paper.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a real-time RGB-D perception pipeline for autonomous hydraulic impact hammers in mining. The pipeline takes synchronized RGB-D frames from two ceiling-mounted stereo cameras plus the hammer's kinematic state, performs self-filtering of the robot from depth maps using approximated link meshes and modeled hydraulic cylinders, fills occluded regions via a depth-based background model, and then segments rocks by two complementary branches: RTMDet instance segmentation in RGB images and geometric point-cloud filtering/clustering. The outputs are prioritized rock-breaking target poses and a robot-free 3D workspace point cloud. The system is implemented on a Jetson AGX Orin using JAX, Open3D, and PyTorch, and is evaluated on a scaled setup consisting of a Bobcat E10 mini-excavator with a hydraulic impact hammer operating over a 1.5x1.5 m steel grate. Reported performance includes roughly 10 Hz output and about 675 ms end-to-end latency for pose generation, quantitative segmentation results for 1–7 rocks, and a real-world closed-loop pressing test in which 72% of 111 reachable target poses resulted in stable contact and a simulated strike via sustained pressure.
Significance. If the claims are upheld, the paper makes a useful systems contribution: it integrates self-filtering, occlusion management, image- and point-cloud-based rock segmentation, surface-normal-based pose generation, and prioritized pose selection into one embedded, real-time pipeline, with a real-world closed-loop validation. The paper is unusually transparent about parameter settings, hardware constraints, and the limitations of the experimental proxy. The main source of concern is not internal inconsistency but the strength of the headline conclusion: the closed-loop experiment measures stable pressing, not actual hammer impacts, so the transition from 'stable contact under sustained pressure' to 'practical applicability for autonomous impact hammer operation' needs either additional experimental evidence or a substantial re-framing.
major comments (3)
- [Section V-C1, Table II; Section VI; Abstract] The central real-world success metric is defined as stable chisel–rock contact while the boom presses with constant current for 5 seconds; the authors explicitly state that this is a 'simulated strike (via sustained pressure exertion)' and that the result 'does not imply that actuating the impact hammer’s end-effector in these situations would result in rock fractures.' Yet the abstract concludes that the system 'is suitable for real-time autonomous impact hammer operation,' and the conclusion states that the 72% success rate 'suggest[s] its practical applicability for autonomous impact hammer operation.' The load-bearing outcome variable is therefore a proxy, and the actual impact dynamics—vibration, rebound, higher slip forces, rock displacement—are never exercised. I would ask for either (i) an additional experiment with real hammer actuation, or (ii) a careful re-scoping of the title
- [Section V-C1, Table II] The 72% success rate is computed after excluding 6 of 117 poses because the reaching controller could not reach them. Since reachability is part of the overall objective ('operationally feasible rock-breaking poses'), the paper should report both the conditional rate (80/111 = 72.07%) and the intention-to-treat rate (80/117 = 68.4%), or at least state clearly that the headline number is conditional on successful reachability. In addition, no confidence interval is given; for 80 successes out of 111 trials, the Wilson 95% interval is approximately 63–80%, so the 72% figure has meaningful uncertainty that should be disclosed.
- [Section III, Section VI, Abstract] All experiments were performed in a laboratory-scale scene: a Bobcat E10 beside a 1.5x1.5 m steel grate with two ceiling-mounted ZED X cameras. The abstract's claim of suitability for 'autonomous impact hammer operation' extrapolates from this scaled setup to full-size underground ore-pass conditions. Differences in grizzly scale, camera placement, dust, lighting, and machine dynamics are not evaluated. This is not an internal inconsistency, but it is a correctness risk for the generalization implied by the headline claim. The paper should either temper the conclusion to the demonstrated scaled setup or add a discussion of which components are expected to transfer and which may require re-validation.
minor comments (5)
- [Section III] Typo: 'the problem consist of' should read 'the problem consists of.'
- [Section V-C1] Missing space in 'across11 different configurations'; should be 'across 11 different configurations.'
- [Appendix D, Table V] The RTMDet node is reported at approximately 15 Hz, but the appendix notes that this is achieved 'artificially' by reusing the most recent inference result. This is important context and should be restated in the main text near the 10 Hz claim, so the reader understands that the 10 Hz pose output includes repeated segmentation results.
- [Table II] The discrepancy between the quantitative 'No contact detected' (27.93%) and qualitative 'No contact detected' (24.32%) is explained only in prose. A brief note in the table caption that the quantitative/qualitative columns use different contact-detection definitions would improve readability.
- [Appendix C, Fig. 20] The architecture diagram is dense and some node names in the text ('pcl_vis_rock_segmentator') do not exactly match the figure labels. Aligning the names would reduce confusion.
Circularity Check
No significant circularity: the central result is an external physical measurement; self-citations are not load-bearing.
full rationale
The paper's derivation chain is empirical rather than a formal derivation. The headline 72% success rate is measured in real-world pressing trials (Table II), with chisel-rock contact detected using a learned dynamical model from the authors' prior work (Leiva et al. 2026) but independently corroborated by human annotations yielding the same success rate. The self-filter uses the robot's kinematic model by design, and the occlusion background model is adapted from external prior work (Yang et al.); these are engineering components, not results derived from the paper's own claims. The RTMDet segmentation model is fine-tuned on a custom dataset from the same testbed, but evaluation is on held-out images and the central pose-generation result does not reduce to segmentation accuracy. Pose-generation thresholds and scoring weights are hand-set and listed in Appendix F; there is no evidence they were fitted to the success-rate outcome. The main caveat is construct validity rather than circularity: the success metric is stable chisel-rock contact during a 5 s simulated press, and the paper explicitly states this 'does not imply that actuating the impact hammer’s end-effector in these situations would result in rock fractures' (Sec. V-C1). Likewise, excluding 6/117 unreachable poses is justified because the control system is independent of the perception pipeline. These are external-validity limitations, not self-referential reductions. The self-citations (Leiva et al. 2026; Cárdenas et al. 2022; Ruiz del Solar et al. 2025) supply components, but the main claims—real-time feasible pose generation at roughly 10 Hz with 72% stable-contact pressing—are independent physical measurements and would stand even if those citations were removed.
Axiom & Free-Parameter Ledger
free parameters (10)
- theta_soft =
0.5618 rad (32.2 deg)
- theta_hard =
0.3618 rad (20.7 deg)
- P_min =
10 points
- scoring_weights_lambda_1_to_4 =
1, 2, 3, 5
- t_max =
15 (~1.5 s at 10 Hz)
- self_filter_box_sphere_offsets =
0.12 m (Fig. 6); 0.05/0.12 m (Fig. 7)
- background_model_thresholds =
d_th, v_th, lambda in Eqs. (12)-(14); not numerically reported
- dbscan_and_voxel_parameters =
not reported
- cross_camera_association_threshold =
not reported
- grizzly_filter_sphere_radius =
not reported
axioms (6)
- domain assumption The scaled Bobcat E10 setup with a 1.5x1.5 m grizzly and ceiling-mounted ZED X cameras is representative of real underground ore-pass impact-hammer environments.
- domain assumption The impact hammer's kinematic model, including the estimated joint offsets, accurately represents the physical machine geometry for self-filtering.
- domain assumption Divergence between a learned dynamical model prediction and the measured configuration reliably indicates chisel-rock contact.
- domain assumption The depth-based background model of Yang et al. (2004, 2005), with the paper's modifications, correctly handles occlusions caused by the moving hammer.
- domain assumption ZED X stereo depth maps, after bilateral filtering and vendor neural post-processing, are accurate enough for rock segmentation and surface-normal analysis.
- standard math Standard computer vision and geometric algorithms (ICP, DBSCAN, SDF equations, pinhole projection) are correct as used.
read the original abstract
Impact hammers, also known as rock-breakers, are essential machines in mining operations, where they perform secondary reduction. In underground mining, these machines are typically teleoperated, limiting operational efficiency. This paper presents a real-time RGB-D perception pipeline as a step towards automating the operation of hydraulic impact hammers used in mining. The proposed system simultaneously generates operationally feasible rock-breaking poses and a robot-free 3D representation of the workspace. The proposed approach combines image-based instance segmentation with geometric point cloud processing, and operates on embedded hardware at approximately 10 Hz with a total latency of around 675 ms, enabling responsive closed-loop behavior when integrated with a control system. Experimental results in a representative scaled scenario demonstrate that the proposed system is suitable for real-time autonomous impact hammer operation.
Figures
Reference graph
Works this paper leans on
-
[1]
2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) , volume=
Real-time multiple objects tracking with occlusion handling in dynamic scenes , author=. 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) , volume=. 2005 , organization=
2005
-
[2]
Proceedings of the ACM 2nd international workshop on Video surveillance & sensor networks , pages=
Real-time and accurate segmentation of moving objects in dynamic scene , author=. Proceedings of the ACM 2nd international workshop on Video surveillance & sensor networks , pages=
-
[3]
Sixth international conference on computer vision (IEEE Cat
Bilateral filtering for gray and color images , author=. Sixth international conference on computer vision (IEEE Cat. No. 98CH36271) , pages=. 1998 , organization=
1998
-
[4]
arXiv preprint arXiv:1801.09847 , year=
Open3D: A modern library for 3D data processing , author=. arXiv preprint arXiv:1801.09847 , year=
-
[5]
James Bradbury and Roy Frostig and Peter Hawkins and Matthew James Johnson and Chris Leary and Dougal Maclaurin and George Necula and Adam Paszke and Jake Vander
-
[6]
Advances in neural information processing systems , volume=
Pytorch: An imperative style, high-performance deep learning library , author=. Advances in neural information processing systems , volume=
-
[7]
arXiv preprint arXiv:2212.07784 , year=
Rtmdet: An empirical study of designing real-time object detectors , author=. arXiv preprint arXiv:2212.07784 , year=
-
[8]
IEEE transactions on information theory , volume=
Least squares quantization in PCM , author=. IEEE transactions on information theory , volume=. 1982 , publisher=
1982
-
[9]
kdd , volume=
A density-based algorithm for discovering clusters in large spatial databases with noise , author=. kdd , volume=
-
[10]
Energies , volume=
Automatic determination of rock-breaking target poses for impact hammers , author=. Energies , volume=. 2022 , publisher=
2022
-
[11]
ICRA workshop on open source software , volume=
ROS: an open-source Robot Operating System , author=. ICRA workshop on open source software , volume=. 2009 , organization=
2009
-
[12]
Applied Sciences , volume=
Haptic teleoperation of impact hammers in underground mining , author=. Applied Sciences , volume=. 2022 , publisher=
2022
-
[13]
Fragblast , volume=
Automatic detection and breaking system for boulders by use of ccd camera and laser pointer , author=. Fragblast , volume=. 1998 , publisher=
1998
-
[14]
Fragblast , volume=
Automatic system for boulder breakage using force sensors , author=. Fragblast , volume=. 1999 , publisher=
1999
-
[15]
Autonomous robotic systems , pages=
Robotics for the mining industry , author=. Autonomous robotic systems , pages=. 2005 , publisher=
2005
-
[16]
IEEE Robotics and Automation Letters , volume=
Learning to break rocks with deep reinforcement learning , author=. IEEE Robotics and Automation Letters , volume=. 2023 , publisher=
2023
-
[17]
arXiv preprint arXiv:2601.07813 , year=
Data-driven control of hydraulic impact hammers under strict operational and control constraints , author=. arXiv preprint arXiv:2601.07813 , year=
-
[18]
2018 Ninth International Conference on Intelligent Control and Information Processing (ICICIP) , pages=
Clustering analysis for secondary breaking using a low-cost time-of-flight camera , author=. 2018 Ninth International Conference on Intelligent Control and Information Processing (ICICIP) , pages=. 2018 , organization=
2018
-
[19]
2019 IEEE 15th International Conference on Automation Science and Engineering (CASE) , pages=
Efficient 3d visual perception for robotic rock breaking , author=. 2019 IEEE 15th International Conference on Automation Science and Engineering (CASE) , pages=. 2019 , organization=
2019
-
[20]
Journal of Field Robotics , volume=
Autonomous robotic rock breaking using a real-time 3D visual perception system , author=. Journal of Field Robotics , volume=. 2021 , publisher=
2021
-
[21]
arXiv preprint arXiv:1804.02767 , year=
Yolov3: An incremental improvement , author=. arXiv preprint arXiv:1804.02767 , year=
-
[22]
2021 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM) , pages=
Robust rock detection and clustering with surface analysis for robotic rock breaking systems , author=. 2021 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM) , pages=. 2021 , organization=
2021
-
[23]
Expert Systems with Applications , volume=
Detecting rocks in challenging mining environments using convolutional neural networks and ellipses as an alternative to bounding boxes , author=. Expert Systems with Applications , volume=. 2022 , publisher=
2022
-
[24]
Sensor fusion IV: control paradigms and data structures , volume=
Method for registration of 3-D shapes , author=. Sensor fusion IV: control paradigms and data structures , volume=. 1992 , organization=
1992
-
[25]
arXiv preprint arXiv:2106.13281 , year=
Brax--a differentiable physics engine for large scale rigid body simulation , author=. arXiv preprint arXiv:2106.13281 , year=
-
[26]
Journal of Open Source Software , volume=
PyVista: 3D plotting and mesh analysis through a streamlined interface for the Visualization Toolkit (VTK) , author=. Journal of Open Source Software , volume=. 2019 , publisher=
2019
-
[27]
2025 , month=dec, note=
Ruiz del Solar, Javier and Leiva Castro, Francisco and Romero Vega, Sandra , title=. 2025 , month=dec, note=
2025
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.