This paper proposes a virtual reality-based dual-mode teleoperation architecture to assist human operators in remotely operating robotic manipulation systems in a safe and flexible way. The architecture, implemented via a finite state machine, enables the operator to switch between two operational modes: the Approach mode, where the operator indirectly controls the robotic system by specifying its target configuration via the immersive Virtual Reality (VR) interface, and the Telemanip mode, where the operator directly controls the robot end-effector motion via input devices. The two independent control modes have been tested along the task of reaching a glass on a table by a sample population of 18 participants. Two working groups have been considered, to distinguish users with previous experience with VR technologies from the novices. The results of the user study presented in this work show the potential of the proposed architecture in terms of usability, both physical and mental workload, and user’s satisfaction. Finally, a statistical analysis showed no significant differences along these three metrics between the two considered groups demonstrating ease of use of the proposed architecture by both people with and with no previous experience in VR.

In this article, we review the main results achieved by the research activities carried out at PRISMA Lab of the University of Naples Federico II where, for 35 years, an interdisciplinary team of experts developed robots that are ultimately useful to humans. We summarize the key contributions made in the last decade in the six research areas of dynamic manipulation and locomotion, aerial robotics, human-robot interaction, artificial intelligence and cognitive robotics, industrial robotics, and medical robotics. After a brief overview of each research field, the most significant methodologies and results are reported and discussed, highlighting their cross-disciplinary and translational aspects. Finally, the potential future research directions identified are discussed.

This paper proposes rigid-body modelling and identification procedures for long-reach dual-arm manipulators in a cable-suspended pendulum configuration. The proposed model relies on a virtually constrained open kinematic chain and lends itself to be simulated through the most commonly used robotic simulators without explicitly account for the cables constraints and flexibility. Moreover, a dynamic parameters identification procedure is devised to improve the simulation model fidelity and reduce the sim-to-real gap for controllers deployment. We show the capability of our model to handle different cable configurations and suspension mechanisms by customising it for two representative cable-suspended dual-arm manipulation systems: the LiCAS arms suspeded by a drone and the CRANEbot system, featuring two Pilz arms suspended by a crane. The identified dynamic models are validated by comparing their evolution with data acquired from the real systems showing a high (between 91.3% to 99.4%) correlation of the response signals. In a comparison performed with baseline pendulum models, our model increases the simulation accuracy from 64.4% to 85.9%. The simulation environment and the related controllers are released as open-source code.

Semi-autonomous telerobotic systems allow both humans and robots to exploit their strengths, while enabling personalized execution of a task. However, for new soft robots with degrees of freedom dissimilar to those of human operators, it is unknown how the control of a task should be divided between the human and robot. This work presents a set of interaction paradigms between a human and a soft growing robot manipulator, and demonstrates them in both real and simulated scenarios. The robot can grow and retract by eversion and inversion of its tubular body, a property we exploit to implement interaction paradigms. We implemented and tested six different paradigms of human-robot interaction, beginning with full teleoperation and gradually adding automation to various aspects of the task execution. All paradigms were demonstrated by two expert and two naive operators. Results show that humans and the soft robot manipulator can split control along degrees of freedom while acting simultaneously. In the simple pick-and-place task studied in this work, performance improves as the control is gradually given to the robot, because the robot can correct certain human errors. However, human engagement and enjoyment may be maximized when the task is at least partially shared. Finally, when the human operator is assisted by haptic feedback based on soft robot position errors, we observed that the improvement in performance is highly dependent on the expertise of the human operator.

This article proposes a Model Predictive Non-Sliding Manipulation (MPNSM) control approach to safely transport an object on a tray-like end-effector of a robotic manipulator. For the considered non-prehensile transportation task to succeed, both non-sliding manipulation and the robotic system constraints must always be satisfied. To tackle this problem, we devise a model predictive controller enforcing sticking contacts, i.e., preventing sliding between the object and the tray, and assuring that physical limits such as extreme joint positions, velocities, and input torques are never exceeded. The combined dynamic model of the physical system, comprising the manipulator and the object in contact, is derived in a compact form. The associated non-sliding manipulation constraint is formulated such that the parametrized contact forces belong to a conservatively approximated friction cone space. This constraint is enforced by the proposed MPNSM controller, formulated as an optimal control problem that optimises the objective of tracking the desired trajectory while always satisfying both manipulation and robotic system constraints. We validate our approach by show- ing extensive dynamic simulations using a torque-controlled 7-degree-of-freedom (DoF) KUKA LWR IIWA robotic manipulator. Finally, demonstrative results from real experiments conducted on a 21-DoF humanoid robotic platform are shown.

This work proposes an operational space control framework for non-prehensile object transportation using a robot arm. The control actions for the manipulator are computed by solving a quadratic programming (QP) problem considering the object's and manipulator's kinematic and dynamic constraints. Given the desired transportation trajectory, the proposed controller generates control commands for the robot to achieve the desired motion whilst preventing object slippage. In particular, the controller minimizes the occurrence of object slippage by adaptively regulating the tray orientation. The proposed approach has been extensively evaluated numerically with a 7-degree-of-freedom manipulator, and it is also verified and validated with a real experimental setup.

Sharing the control of a robotic system with an autonomous controller allows a human to reduce his/her cognitive and physical workload during the execution of a task. In recent years, the development of inference and learning techniques has widened the spectrum of applications of shared control (SC) approaches, leading to robotic systems that are capable of seamless adaptation of their autonomy level. In this perspective, shared autonomy (SA) can be defined as the design paradigm that enables this adapting behavior of the robotic system. This letter collects the latest results achieved by the research community in the field of SC and SA with special emphasis on physical human-robot interaction (pHRI). Architectures and methods developed for SC and SA are discussed throughout the paper, highlighting the key aspects of each methodology. A discussion about open issues concludes this letter.

This article proposes a shared-control teleoperation architecture for robot manipulators transporting an object on a tray. Differently from many existing studies about remotely operated robots with firm grasping capabilities, we consider the case in which, in principle, the object can break its contact with the robot end-effector. The proposed shared-control approach automatically regulates the remote robot motion commanded by the user and the end-effector orientation to prevent the object from sliding over the tray. Furthermore, the human operator is provided with haptic cues informing about the discrepancy between the commanded and executed robot motion, which assist the operator throughout the task execution. We carried out trajectory tracking experiments employing an autonomous 7 degree-of-freedom (DoF) manipulator and compared the results obtained using the proposed approach with two different control schemes (i.e., constant tray orientation and no motion adjustment). We also carried out a human-subjects study involving eighteen participants, in which a 3-DoF haptic device was used to teleoperate the robot linear motion and display haptic cues to the operator. In all experiments, the results clearly show that our control approach outperforms the other solutions in terms of sliding prevention, robustness, commands tracking, and user’s preference.

The most effective expression of the 4.0 Era is represented by cyber-physical systems (CPSs). Historically, measurement and monitoring systems (MMSs) have been an essential part of CPSs; however, by introducing the 4.0 enabling technologies into MMSs, a MMS can evolve into a cyber-physical measurement system (CPMS). Starting from this consideration, this work reports a preliminary case study of a CPMS, namely an innovative bioinspired robotic platform that can be used for measurement and monitoring applications in confined and constrained environments. The innovative system is a “soft growing” robot that can access a remote site through controlled lengthening and steering of its body via a pneumatic actuation mechanism. The system can be endowed with different sensors at the tip, or along its body, to enable remote measurement and monitoring tasks; as a result, the robot can be employed to effectively deploy sensors in remote locations. In this work, a digital twin of the system is developed for simulation of a practical measurement scenario. The ultimate goal is to achieve a self-adapting, fully/partially autonomous system for remote monitoring operations to be used reliably and safely for the inspection of unknown and/or constrained environments.

In this paper, an intelligent variable impedance control combined with a fuzzy gain dynamic surface is proposed to improve the interaction of the robot manipulator with an unknown varied environment. The parameters of the proposed variable impedance are adapted by optimization an introduced cost function using a recurrent fuzzy wavelet network. The stability conditions for the varying inertial, stiffness and damping are presented to guarantee the stability of the variable impedance. Additionally, a fuzzy dynamic surface method is developed to tune the gains of the dynamic surface as a robust controller. The proposed fuzzy gain dynamic surface is used to force the end-effector of the manipulator to track the desired impedance profile in the presence of large disturbances. Using Lyapunov's method, the stability of the mentioned closed-loop system is proved. Finally, by using a designed simulator for IRB120 (ABB) robot, several simulations are carried out to verify the performance of the proposed method for the execution of various tasks in an unknown varied environment in the presence of large disturbances.

Although substantial progresses have been made in robot-assisted laparoscopic surgery, the graspers for existing sur- gical systems generally remain non-sensorized forceps design with limited functions. This paper presents the design, development and preliminary evaluation of the MUSHA Hand II, a multi- functional hand with force sensors for robot-assisted laparoscopic surgery. The proposed hand has three snake-like underactuated fingers that can be folded into a φ12 mm cylindrical form. Each finger has a three-axis force sensor, to provide force information. After been deployed into an abdominal cavity, the hand can be configured to either grasper mode, retractor mode or palpation mode for different tasks. Underactuated finger design enhances the adaptivity in grasping and the compliance in interaction with the environment. In addition, fingertip force sensors can be utilized for palpation to obtain a real-time stiffness map of organs. Using the da Vinci Research Kit (dVRK) as a robotic testbed, the functionality of the hand has been demonstrated and experiments have been conducted, including robotic palpation and organ manipulation. The results suggest that the hand can effectively enhance the functionality of a robotic surgical system and overcome the limits on force sensing introduced by the use of robots in laparoscopic surgery.

The da Vinci Research Kit (dVRK) is a first generation da Vinci robot repurposed as a research platform and coupled with software and controllers developed by research users. An already quite wide community is currently sharing the dVRK (32 systems in 28 sites worldwide). The access to the robotic system for training surgeons and for developing new surgical procedures, tools and new control modalities is still difficult due to the limited availability and high maintenance costs. The development of simulation tools provides a low cost, easy and safe alternative to the use of the real platform for preliminary research and training activities. The Portable dVRK, which is described in this work, is based on a V-REP simulator of the dVRK patient side and endoscopic camera manipulators which are controlled through two haptic interfaces and a 3D viewer, respectively. The V-REP simulator is augmented with a physics engine allowing to render the interaction of new developed tools with soft objects. Full integration in the ROS control architecture makes the simulator flexible and easy to be interfaced with other possible devices. Several scenes have been implemented to illustrate performance and potentials of the developed simulator.

Keyhole surgery is characterized by loss of dexterity of surgeon's movements because of the limited workspace, nonintuitive motor skills of the surgical systems, and loss of tactile sensation that may lead to tissue damage and bad execution of the tasks. In this paper, a three-fingered underactuated miniature tool for robot-aided laparoscopic surgery is presented. The design is conceived to realize a closed-hand configuration allowing the insertion of the tool into the abdominal cavity through the trocar in one step and to reach different grasping as well as pushing/holding configurations once in the cavity. Aiming to replicate human hand dexterity and versatility, different solutions for the kinematic structure of the hand are analyzed using quality indices to evaluate the manipulability and stability of the grasp. Furthermore, a first prototype of fingertip force sensor based on fiber Bragg grating (FBG) technology has been realized and tested. The design choices of the prototype are described and discussed with the aid of experiments. The whole concept and the need for such anthropomorphic tool are discussed with surgeons to highlight constraints and potentials in surgical tasks. The feedback by expert surgeons is used to provide specifications and improvements to the kinematics and mechanical design. The investigations of different designs allow identifying the optimal solution to improve grasping and manipulation capabilities. The tests on FBG sensors led to the conclusion that this technology guarantees good performance and can be a good solution for applications in surgical robotics.

Industry 4.0 demands the heavy usage of robotic mobile manipulators with high autonomy and intelligence. The goal is to accomplish dexterous manipulation tasks without prior knowledge of the object status in unstructured environments. It is important for the mobile manipulator to recognize and detect the objects, determine manipulation pose, and adjust its pose in the workspace fast and accurately. In this research, we developed a stereo vision algorithm for the object pose estimation using point cloud data from multiple stereo vision systems. An improved iterative closest point algorithm method is developed for the pose estimation. With the pose input, algorithms and several criteria are studied for the robot to select and adjust its pose by maximizing its manipulability on a given manipulation task. The performance of each technical module and the complete robotic system is finally shown by the virtual robot in the simulator and real robot in experiments. This study demonstrates a setup of autonomous mobile manipulator for various flexible manufacturing and logistical scenarios.

We propose novel haptic guidance methods for a dual-arm telerobotic manipulation system, which are able to deal with several different constraints, such as collisions, joint limits, and singularities. We combine the haptic guidance with shared-control algorithms for autonomous orientation control and collision avoidance meant to further simplify the execution of grasping tasks. The stability of the overall system in various control modalities is presented and analyzed via passivity arguments. In addition, a human subject study is carried out to assess the effectiveness and applicability of the proposed control approaches both in simulated and real scenarios. Results show that the proposed haptic-enabled shared-control methods significantly improve the performance of grasping tasks with respect to the use of classic teleoperation with neither haptic guidance nor shared control.

During robot-aided surgical interventions, the surgeon can be benefitted from the application of virtual fixtures (VFs). Though very effective, this technique is very often not practicable in unstructured surgical environments. In order to comply with the environmental deformation, both the VF geometry and the constraint enforcement parameters need to be online defined/adapted. This letter proposes a strategy for an effective use of VF assistance in minimally invasive robotic surgical tasks. An online VF generation technique based on the interaction force measurements is presented. Pose and geometry adaptations of the VF are considered. Passivity of the overall system is guaranteed by using energy tanks passivity-based control. The proposed method is validated through experiments on the da Vinci Research Kit.

In laparoscopic minimally invasive robotic surgery, a teleoperated robot is interposed between the patient and the surgeon. Despite the robot aid, the manipulation capabilities of surgical instruments are far from those of the human hand. In this letter, we want to make a step forward toward robotic solutions that can improve manipulation capabilities of the surgical instruments. A new concept of needle-driver tool is presented, which takes inspiration from the human hand model. The idea is to modify a standard laparoscopic tool by introducing an additional degree of freedom, which allows in-hand reorientation of the suturing needle. A 3D printed prototype has been built to validate the tool design. The improved manipulation capabilities have been assessed quantitatively by evaluating a weighted dexterity index along a single stitch trajectory. Moreover, a comparison between our tool and a standard needle driver has been done in terms of time required for the execution of a complete suturing sequence.

This paper presents a novel instant 3D whole body scanner for healthcare applications. It is based on photogrammetry, a digital technology which allows to reconstruct the surface of objects starting from multiple pictures. The motivation behind this work is the development of minimally invasive procedures for instant data acquisitions of anatomical structure. The scanner provides several features of interests in 3D body scanning technologies for the healthcare domains: (i) instant capture of human body models; (ii) magnitude of accuracy in the order of 1 mm; (iii) simplicity of use; (iv) possibility to scan using different settings; (v) possibility to reconstruct the texture. The system is built upon a modular and distributed architecture. In this paper we highlight its key concepts and the methodology which has led to the current product. We illustrate its potential through one of the most promising 3D scanning healthcare applications: the data acquisition and processing of human body models for the digital manufacturing process of prostheses and orthoses. We validate the overall system in terms of conformity with the the initial requirements.

The minimization of the turnaround time, the duration which an aircraft must remain parked at the gate, is an important goal of airlines to increase their profitability. This work introduces a procedure to minimize of the turnaround time by speeding up the boarding time in passenger aircrafts. This is realized by allocating the seat numbers adaptively to passengers when they pass the boarding gate and not before. Using optical sensors, an agility measure is assigned to each person and also a measure to characterize the size of her/his hand-luggage. Based on these two values per passenger and taking into account additional constraints, like reserved seats and the belonging to a group, a novel seat allocation algorithm is introduced to minimize the boarding time. Extensive simulations show that a mean reduction of the boarding time with approximately 15% is achieved compared to existing boarding strategies. The costs of introducing the proposed procedure are negligible, while the savings of reducing the turnaround time are enormous, considering that the costs generated by inactive planes on an airport are estimated to be about 30 $ per minute.