pith. sign in

arxiv: 2506.23723 · v2 · submitted 2025-06-30 · 💻 cs.RO

A comprehensive control architecture for semi-autonomous dual-arm robots in agriculture settings

Jozsef Palmieri , Paolo Di Lillo , Stefano Chiaverini , Alessandro Marino This is my paper

Pith reviewed 2026-05-19 07:52 UTC · model grok-4.3

classification 💻 cs.RO
keywords hierarchical quadratic programmingdual-arm robotagricultural roboticsgrape harvestingsemi-autonomous controlinteraction forcesvineyard robotics
0
0 comments X

The pith

A single Hierarchical Quadratic Programming framework controls a 16-DOF dual-arm robot for grape harvesting, force management, and semi-autonomous assistance in vineyards.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops a unified control architecture for a mobile dual-arm robot to carry out grape harvesting in vineyards. Hierarchical Quadratic Programming manages multiple tasks at different priority levels, covering robot limits, motion to reach perception-selected bunches, and human operator inputs. The same optimization structure is used to regulate interaction forces arising from perception uncertainty in the unstructured environment. This enables both fully autonomous harvesting and semi-autonomous mode where a human assists when needed. The architecture is demonstrated first in laboratory tests and then in real vineyard conditions.

Core claim

The authors show that Hierarchical Quadratic Programming can be used to control a 16 degree-of-freedom dual-arm mobile robot to harvest grape bunches chosen by a perception system. The method handles equality and inequality constraints at multiple priority levels. The same controller is applied to manage interaction forces that arise from perception errors, and it supports semi-autonomous mode where a human helps complete the task.

What carries the argument

Hierarchical Quadratic Programming solver that stacks and resolves tasks by priority to satisfy equality and inequality constraints at once.

If this is right

  • The robot executes harvesting while respecting joint limits and avoiding self-collisions.
  • Interaction forces are regulated inside the same priority stack used for motion tasks.
  • Semi-autonomous mode lets a human supply corrective inputs that the controller incorporates at lower priority.
  • The architecture was shown to work in both controlled lab settings and unstructured vineyard rows.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The priority-based solver could be reused for other field tasks such as pruning or selective spraying.
  • Stronger perception would lower the frequency of required human assistance.
  • The approach may extend to other mobile manipulators operating in cluttered outdoor settings.

Load-bearing premise

The perception system must deliver grape selections accurate enough for the controller to avoid collisions and regulate forces without extra safety layers or constant human oversight.

What would settle it

A vineyard trial in which the robot autonomously follows a perception-selected grape target yet collides with the vine or environment or fails to complete the harvest.

read the original abstract

The adoption of mobile robotic platforms in complex environments, such as agricultural settings, requires these systems to exhibit a flexible yet effective architecture that integrates perception and control. In such scenarios, several tasks need to be accomplished simultaneously, ranging from managing robot limits to performing operational tasks and handling human inputs. The purpose of this paper is to present a comprehensive control architecture for achieving complex tasks such as robotized harvesting in vineyards within the framework of the European project CANOPIES. In detail, a 16-DOF dual-arm mobile robot is employed, controlled via a Hierarchical Quadratic Programming (HQP) approach capable of handling both equality and inequality constraints at various priorities to harvest grape bunches selected by the perception system developed within the project. Furthermore, given the complexity of the scenario and the uncertainty in the perception system, which could potentially lead to collisions with the environment, the handling of interaction forces is necessary. Remarkably, this was achieved using the same HQP framework. This feature is further leveraged to enable semi-autonomous operations, allowing a human operator to assist the robotic counterpart in completing harvesting tasks. Finally, the obtained results are validated through extensive testing conducted first in a laboratory environment to prove individual functionalities, then in a real vineyard, encompassing both autonomous and semi-autonomous grape harvesting operations.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript presents a control architecture for a 16-DOF dual-arm mobile robot in vineyard grape harvesting under the CANOPIES project. It uses Hierarchical Quadratic Programming (HQP) to simultaneously enforce equality and inequality constraints at multiple priority levels for task execution, joint limits, collision avoidance, and interaction-force management. The same HQP stack is extended to support semi-autonomous human-assisted operations when perception uncertainty risks collisions. Validation consists of laboratory tests of individual modules followed by vineyard trials of both fully autonomous and semi-autonomous harvesting.

Significance. A well-supported demonstration that a single HQP stack can integrate perception-driven task selection, force-aware safety, and human-in-the-loop assistance in unstructured agricultural settings would be a useful engineering contribution to field robotics. The approach avoids ad-hoc switching between controllers and could reduce the need for separate safety layers if the force inequalities demonstrably absorb typical perception noise.

major comments (2)
  1. [Vineyard results section] Vineyard results section: the text states that the system performed autonomous and semi-autonomous harvesting but reports no per-trial quantitative metrics (success rate per bunch, peak measured contact force, number of force-limit activations, or human interventions required). Without these numbers it is impossible to verify the central claim that the HQP inequality constraints reliably compensate for perception localization or classification errors.
  2. [HQP formulation] HQP formulation (likely §3): the priority ordering and the exact inequality constraints used for interaction-force bounds are described only at a high level. Explicit equations showing how force limits are inserted as lower-priority inequalities and how they interact with perception-driven task equalities are needed to assess whether the stack can absorb the expected level of vine-structure uncertainty.
minor comments (2)
  1. [Figures] Figure captions for the vineyard setup should explicitly label the 16-DOF configuration and the coordinate frames used for grape selection.
  2. [Results] A short table summarizing laboratory versus vineyard performance (even if only qualitative) would improve readability.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed and constructive review of our manuscript on the HQP-based control architecture for semi-autonomous dual-arm harvesting. We value the recognition of the engineering contribution in integrating perception, force management, and human assistance within a single hierarchical framework. We address each major comment below and will revise the manuscript to incorporate the suggested improvements.

read point-by-point responses

  1. Referee: [Vineyard results section] Vineyard results section: the text states that the system performed autonomous and semi-autonomous harvesting but reports no per-trial quantitative metrics (success rate per bunch, peak measured contact force, number of force-limit activations, or human interventions required). Without these numbers it is impossible to verify the central claim that the HQP inequality constraints reliably compensate for perception localization or classification errors.

    Authors: We agree that the absence of per-trial quantitative metrics limits the ability to fully verify the claims regarding perception-error compensation. In the revised manuscript we will add a dedicated results table (or expanded subsection) reporting success rate per harvested bunch, peak measured contact forces, number of force-limit activations, and count of human interventions across the autonomous and semi-autonomous vineyard trials. These data were collected during the CANOPIES field experiments and will be presented with appropriate statistical summaries to directly support the effectiveness of the force inequalities. revision: yes

  2. Referee: [HQP formulation] HQP formulation (likely §3): the priority ordering and the exact inequality constraints used for interaction-force bounds are described only at a high level. Explicit equations showing how force limits are inserted as lower-priority inequalities and how they interact with perception-driven task equalities are needed to assess whether the stack can absorb the expected level of vine-structure uncertainty.

    Authors: We concur that a more explicit mathematical presentation is required for reproducibility and assessment. In the revised Section 3 we will insert the full HQP optimization problem at each priority level, explicitly stating the ordering (joint limits and self-collision avoidance at the highest level, followed by end-effector task equalities for bunch grasping, then interaction-force inequalities at the subsequent level). The force bounds will be written as ||f_ext|| ≤ f_max (with component-wise limits where appropriate) and shown as lower-priority inequality constraints that are activated only when higher-priority equalities cannot be satisfied exactly, thereby allowing the solver to absorb vine-structure localization uncertainty without violating safety constraints. revision: yes

Circularity Check

0 steps flagged

No circularity in HQP control architecture integration

full rationale

The paper describes an engineering integration of standard Hierarchical Quadratic Programming (HQP) methods to handle equality/inequality constraints, interaction forces, and semi-autonomous operations for a 16-DOF dual-arm robot. No equations, derivations, or predictions are presented that reduce by construction to fitted inputs, self-definitions, or self-citation chains. The central claims rest on applying established HQP techniques to vineyard harvesting tasks, with experimental validation in lab and field settings, rendering the derivation chain self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The architecture rests on the standard mathematical properties of Hierarchical Quadratic Programming for prioritized constraint satisfaction and on the assumption that interaction forces can be treated as inequality constraints within the same solver.

axioms (1)
  • domain assumption Hierarchical Quadratic Programming can simultaneously satisfy equality and inequality constraints at multiple priority levels without infeasibility in the target scenarios.
    Invoked when stating that the HQP approach handles robot limits, operational tasks, and human inputs.

pith-pipeline@v0.9.0 · 5763 in / 1239 out tokens · 48652 ms · 2026-05-19T07:52:25.733388+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.