Combined Vision and Wearable System for Daily Activity Recognition

Social assistive robotics aims at improving the quality of life of elderly people and caregivers. Human Activity Recognition (HAR) AQ1 is one of the capabilities the assistive robot should be endowed with, to allow aged people to independently live in their homes. This work deals with the problem of performing HAR by employing two wearable inertial sensors and one RGB-D camera, mounted on the social robot Pepper. Specifically, the main purpose is to prove that Pepper robot is able to correctly recognize daily living activities by exploiting the information coming from the RGB-D camera and one inertial sensor placed on the index finger of the subject. Ten users were asked to perform ten activities while wearing an inertial glove, SensHand, and while being recorded by the camera. Two different perspectives of the robot were studied to understand if a good activity recognition could be obtained when the robot is in front of the person and on his side. The results show that almost the same recognition performances are obtained when combining the visual sensor, no matter the chosen perspective, with the inertial sensor only on the index (95%), with respect to the fusion of the same camera with the inertial sensor on the index and on the wrist (96%). This supports the conclusion that elderly people could just wear a small ring on the index finger to allow the robot to recognize their activities, taking advantage from a system which is comfortable and easy-to-wear.