Q-learning-based Model-free Swing Up Control of an Inverted Pendulum

An inverted pendulum is a high non-linear, chaotic and dynamically complex system, which presents problems for traditional controllers that require feedback loops and a precise dynamic model of the system. Reinforcement learning is an promising approach, since it does not need the dynamic model and generates autonomous actions based on experience. However, solving a control problem with reinforcement learning is challenging, because every dynamic system has a continuous state space. In this paper, an algorithm that uses Q-learning with function approximation is proposed to control an inverted pendulum. The algorithm consists of two stages, one for swing up, and another for the control at upright position. Results show that the proposed approach reaches the control objectives.