Reinforcement learning control for a three-link biped robot with energy-efficient periodic gaits

Designing a high-performance controller for the walking gaits of biped robots remains an open research area due to their strong nonlinearity and non-smooth responses. To overcome such challenges, a humanoid robot with a torso, i.e., a three-link biped robot involving both impact and friction, is developed firstly. Then, the twin delayed deep deterministic policy gradient algorithm is adopted to design the reinforcement learning controller for the proposed biped robot. For the specified control targets, i.e., energy-efficient periodic gaits for both the downhill and uphill cases, a reward function utilizing the Poincare map and the power function is constructed to provide guidelines for the controller. Thus, the proposed controller can learn to adaptively output accurate cosine torques to achieve the goal without relying on the pre-designed reference trajectories or embedded unstable periodic gaits. A comparative study between the proposed reinforcement learning and neural network proportion differentiation controllers demonstrates the proposed controller can lead to accurate and energy-efficient periodic gaits and provide strong adaptability and robustness within a wide variety of walking slopes.