A RGB-D camera-based approach for robot arm-hand teleoperated control

In this paper a master-slave teleoperation system is proposed for reach-and-grasp operations in hazardous environments. A RGB-D camera and a robust marker-based Bayesian estimation method is proposed for tracking the human hand joint positions on the master side. Accuracy and repeatability of the estimation algorithm have been measured through the comparison with the data acquired through optoelectronic system. An anthropomorphic arm-hand robotic system is used on the slave side. The human operator can select the robot working area and the type of grasp action through finger motion. On the other hand, the human wrist motion is mapped into the robot workspace in order to teleoperate the robot end effector during the reaching phase. The optimal robot grasp configuration is automatically performed by the slave robot through a grasp synthesis algorithm previously proposed by the authors. The proposed approach has been experimentally validated on the KUKA LWR 4+ robot arm and the DLR-HIT Hand II.