| Planning and optimization methods have been widely applied to the problem of trajectory generation for autonomous robotics. The performance of such methods, however, is critically dependent on the choice of objective function being optimized, and is non-trivial to design. On the other end of the spectrum, efforts on learning autonomous behavior from user-provided demonstrations have largely been focused on reproducing behavior similar in appearance to the provided demonstrations. An alternative approach, known as Inverse Reinforcement Learning (IRL), is to learn an objective function that the demonstrations are assumed to be optimal under. With the help of a planner or trajectory optimizer, such an approach allows the system to synthesize novel behavior in situations that were not experienced in the demonstrations.;We present novel algorithms for IRL that have successfully been applied in two real-world, competitive robotics settings: (1) In the domain of rough terrain quadruped locomotion, we present an algorithm that learns an objective function for foothold selection based on "terrain templates". The learner automatically generates and selects the appropriate features which form the objective function, which reduces the need for feature engineering while attaining a high level of generalization. (2) For the domain of autonomous manipulation, we present a probabilistic model of optimal trajectories, which results in new algorithms for inverse reinforcement learning and trajectory optimization in high-dimensional settings. We apply this method to two problems in robotic manipulation: redundancy resolution in inverse kinematics, and trajectory optimization for grasping and manipulation. Both methods have proven themselves as part of larger integrated systems in competitive settings against other teams, where testing was conducted by an independent test team in situations that were not seen during training. |
