Towards socially intelligent robots in human centered environment
Robots are no longer going to be isolated machines working in factory or merely research platforms used in controlled lab environment. Very soon, robots will be the part of our day-to-day lives. Whether it is street, o ce, home or supermarket, robots will be there to assist and serve us. For such robots to be accepted and appreciated, they should explicitly consider the presence of human in all their planning and decision making strategies, whether it is for motion, manipulation or interaction. This thesis explores various socio-cognitive aspects ranging from perspective-taking, social navigation behaviors, cooperative planning, proactive behaviors to learning task semantics from demonstration. Further, by identifying key ingredients of these aspects, we equipped the robots with basic socio-cognitive intelligence, as a step towards making the robots to co-exist with us in complete harmony. In the context of socially acceptable navigation of a robot, it is a must that the robot should no longer treat us, the human, only as dynamic obstacles in the environment. For example, the robot should even decide to take a longer path, if it is satisfying the human's desire and expectation and not creating any confusion, fear, anger or surprise by its motion. This requires the robot to be able to reason about various criteria ranging from clearance, environment structure, unknown objects, social conventions, proximity constraints, presence of an individual or a group of people, etc. Similarly, for the task when the robot has to guide a person from his/her current position to another place, it should support the person's activities and guide him/her in the way he/she wants to be guided. It is quite natural that there will be intentional or unintentional deviations in the person's motion from the path expected by robot. Further, because of person's behavior of leave-taking or temporary suspending the guiding process, if required, the robot should exhibit goal oriented approaching and re-engagement behaviors. A human friendly robot should neither be over-reactive nor be simple wait and move machine. On the other hand, when a robot has to explicitly work together with us in a cooperative Human-Robot Interactive manipulation scenario, it should be able to analyze various abilities and a ordances of the person it is interacting with. Such capabilities of perspective taking is important for various decisions e.g. where to put an object so that human can reach it with least e ort, where and how to show an object to the human, how to grasp an object so that human can also grasp it for object hand-over tasks, etc. All these require the robot to reason beyond the stability of object's grasp and placement even for basic tasks such as show, give, hide make-accessible, put away, etc. Capability to ground day-to-day interaction with the human, to ground the changes in the environment, which happened in the absence of the robot, to generate a shared plan for solving day-to-day tasks, such as clean the table, are some of the other important aspects for the existence of the robots in our day-to-day life. The grounding could be in terms of the object that the human is trying to refer, the agents and the actions, which might be responsible behind some changes, whereas the task planning could be deciding possible cooperation and help among di erent agents. All these requires the robot to reason at di erent levels for planning the task: at symbolic level to decide how to achieve the task and to assign roles to the agents; at geometric level to ensure the feasibility of the actions. Further, reasoning on the e orts and current state and desire of the agents should be taken into account to decide about the amount, extent and method of cooperation, and for grounding interaction and changes. Another aspect of socio-cognitive interaction is behaving proactively, i.e. planning and acting in advance by anticipating the future needs, problems or changes.This demands the robots to be capable of reasoning about how to behave proactively, where to behave proactively to support ongoing interaction or task and so on. Learning from demonstration of day-to-day tasks is an important aspect for the robot to e ciently perform the tasks. Even for basic tasks such as give, hide, make accessible, show, etc., depending upon the situation, the same task could be performed entirely di erently. We should not expect that for each and every task, the robot will be provided with a situation-by-situation based example about how to perform that task. Hence, just imitating the actions of a demonstration is not su cient. The robot should be able to understand the goal of the demonstration, i.e. what does the task mean in terms of desired e ect. The robot should learn it autonomously at appropriate level of abstraction to be able to reproduce them, in diverse situations in di erent ways. It requires reasoning beyond the levels of trajectory and sub-actions. This thesis focuses on these issues, which raise new challenges that cannot be handled appropriately by simple adaptation of state of the art robotics planning, control and decision making techniques. The thesis, rst identi es such basic socio-cognitive ingredients from the child development and human behavioral psychology research and presents the general architecture for socially intelligent human-robot interaction. Next, we will present a generalized domain theory for Human Robot Interaction (HRI) and derive various research challenges under a uni ed framework. Further, we will introduce new terms and concepts from HRI point of view and develop frameworks for integrating them in robot's motion, manipulation and interaction behaviors. Implementation results on di erent types of real robots (PR2, HRP2, Jido,...) will show the proof of concept. This is a step towards Socially Intelligent Robots with the vision to build a base for developing more complex socio-cognitive robot behaviors for future co-existence of human and robot in complete harmony.