Integration of an autonomous system with human-in-the-loop for grasping an unreachable object in the domestic environment

In recent years, autonomous robots have proven capable of solving tasks in complex environments. In particular, robot manipulations in activities of daily living (ADL) for service robots have been widely developed. However, manipulations of grasping an unreachable object in domestic environments still present difficulty. To perform those applications better, we developed an autonomous system with human-in-the-loop that combined the cognitive skills of a human operator with autonomous robot behaviors. In this work, we present techniques for integration the system for assistive mobile manipulation and new strategies to support users in the domestic environment. We demonstrate that the robot can grasp multiple objects with random size at known and unknown table heights. Specifically, we developed three strategies for manipulation. We also demonstrated these strategies using two intuitive interfaces, a visual interface in rviz and a voice user interface with speech recognition. Moreover, the robot can select strategies automatically in random scenarios, which make the robot intelligent and able to make decisions independently in the environment. We demonstrated that our robot shows the capabilities for employment in domestic environments to perform actual tasks.