Stefan Schaal
Professor of Computer Science, Neuroscience, and Biomedical Engineering, University of Southern California
March 28, 2014
Abstract
Controlling a complex movement system requires making perceptual and control decisions at every moment of time, and learning and adaptation to improve the system's performance. High dimensional continuous state-action spaces still pose significant scaling problems for learning algorithms to find (approximately) optimal solutions, and appropriate task descriptions or cost functions require a large amount of human guidance. In order to address autonomous skillful movement generation in complex robot and task scenarios, we have been working on a variety of subproblems to facilitate robust task achievement. Among these topics are general representations for movement in form of movement primitives, trajectory-based reinforcement learning with path integral reinforcement learning, and inverse reinforcement learning to extract the "intent" of observed behavior. However, this "action centric" view of skill acquisition needs to be extended with a stronger perceptual component, as, in the end, it is the entire perception-action-learning loop that could be considered the key element to address, rather than isolated components of this loop. In some tentative initial research, we have been exploring Associative Skill Memories, i.e., the simple idea to start memorizing all sensory events and their statistics together with each movement skill. This concepts opens a wide spectrum of adding predictive, corrective, and switching behaviors in motor skills, and may create an interesting foundation to automatically generate the graphs underlying complex sequential motor skills. Our research results will be illustrated in various experiments with complex anthropomorphic robot systems.
Speaker Biography
Stefan Schaal is Professor of Computer Science, Neuroscience, and Biomedical Engineering at the University of Southern California, and a Founding Director of the Max-Planck-Insitute for Intelligent Systems in Tuebingen, Germany. He is also an Invited Researcher at the ATR Computational Neuroscience Laboratory in Japan, where he held an appointment as Head of the Computational Learning Group during an international ERATO project, the Kawato Dynamic Brain Project (ERATO/JST). Dr. Schaal's research interests include topics of statistical and machine learning, neural networks, computational neuroscience, functional brain imaging, nonlinear dynamics, nonlinear control theory, and biomimetic robotics. He applies his research to problems of artificial and biological motor control and motor learning, focusing on both theoretical investigations and experiments with human subjects and anthropomorphic robot equipment.
