Towards an Interactive Human-Robot Relationship Developing a Customized Robot's Behaviour to Human's Profile

Robots become more and more omnipresent in our life and society, and many challenges arise when we try to use them in a social context. This thesis focuses on how to generate an adapted robot's behavior to human's profile so as to enhance the human-robot relationship. This research addresses a wide range of complex problems varying from analyzing and understanding human's emotion and personality to synthesizing a complete synchronized multimodal behavior that combines gestures, speech, and facial expressions. Our methodologies have been examined experimentally with NAO robot from Aldebaran Robotics and ALICE robot from Hanson Robotics. The first part of this thesis focuses on emotion analysis and discusses its evolutionary nature. The fuzzy nature of emotions imposes a big obstacle in front of defining precise membership criteria for each emotion class. Therefore, fuzzy logic looks appropriate for modeling these complex data sets, as it imitates human logic by using a descriptive and imprecise language in order to cope with fuzzy data. The variation of emotion expressivity through cultures and the difficulty of including many emotion categories inside one database, makes the need for an online recognition system of emotion as a critical issue. A new online fuzzy-based emotion recognition system through prosodic cues was developed in order to detect whether the expressed emotion confirms one of the previously learned emotion clusters, or it constitutes a new cluster (not learned before) that requires a new verbal and/or nonverbal action to be synthesized. On the other hand, the second part of this thesis focuses on personality traits, which play a major role in human social interaction. Different researches studied the long term effect of the extraversion-introversion personality trait on human's generated multimodal behavior. This trait can, therefore, be used to characterize the combined verbal and nonverbal behavior of a human interacting with a robot so as to allow the robot to adapt its generated multimodal behavior to the interacting human's personality. This behavior adaptation could follow either the similarity attraction principle (i.e., individuals are more attracted by others who have similar personality traits) or the complementarity attraction principle (i.e., individuals are more attracted by others whose personalities are complementary to their own personalities) according to the context of interaction. In this thesis, we examine the effects of the multimodality and unimodality of the generated behavior on interaction, in addition to the similarity attraction principle as it considers the effect of the initial interaction between human and robot on the developing relationship (e.g., friendship), which makes it more appropriate for our interaction context. The detection of human's personality trait as being introverted or extraverted is based on a psycholinguistic analysis of human's speech, upon which the characteristics of the generated robot's speech and gestures are defined. Last but not least, the third part of this thesis focuses on gesture synthesis. The generation of appropriate head-arm metaphoric gestures does not follow a specific linguistic analysis. It is mainly based on the prosodic cues of human's speech, which correlate firmly with emotion and the dynamic characteristics of metaphoric gestures. The proposed system uses the Coupled Hidden Markov Models (CHMM) that contain two chains for modeling the characteristic curves of the segmented speech and gestures. When a speech-test signal is present to the trained CHMM, a corresponding set of adapted metaphoric gestures will be synthesized. An experimental study (in which the robot adapts the emotional content of its generated multimodal behavior to the context of interaction) is set for examining the emotional content of the generated robot's metaphoric gestures by human's feedback di- rectly. Besides, we examine the effects of both the generated facial expressions using the expressive face of ALICE robot, and the synthesized emotional speech using the text to speech toolkit (Mary-TTS) on enhancing the expressivity of the robot, in addition to comparing between the effects of the multimodal interaction and the interaction that employs less affective cues on human. Generally, the research on understanding human's profile and generating an adapted robot's behavior opens the door to other topics that need to be addressed in an elaborate way. These topics include, but not limited to: developing a computational cognitive architecture that can simulate the functionalities of the human brain areas that allow understanding and generating speech and physical actions appropriately to the context of interaction, which constitutes a future research scope for this thesis.