Active compliance control of robot peg-in-hole assembly based on combined reinforcement learning

Robot peg-in-hole assembly has always been a research hotspot. With the application of reinforcement learning in the field of robotics in recent years, the assembly strategy based on reinforcement learning has gained significant attention. However, the application of reinforcement learning in physical robots is still limited due to its requirement of a large number of training episodes to explore the environment. In this paper, a novel combined reinforcement learning method is proposed. Firstly, a state judgment Deep Q-Network (SDQN) and a parameter optimization Deep Q-Network (PDQN) are integrated into a framework to train a variable impedance controller. The SDQN strategy can judge the contact state and then select the parameter range for optimization. Then, the PDQN module can generate parameters of the variable impedance controller. Secondly, a reward function for SDQN is designed by incorporating the posture error and insertion depth into the calculation of the negative reward value to ensure assembly speed. The experiment is conducted with a clearance of 0.40 mm, resulting in a success rate of 96% for the assembly task. These findings validate the effectiveness of the proposed combined reinforcement learning method.