Study on force control for robot massage with a model-based reinforcement learning algorithm

When a robot end-effector contacts human skin, it is difficult to adjust the contact force autonomously in an unknown environment. Therefore, a robot force control algorithm based on reinforcement learning with a state transition model is proposed. In this paper, the dynamic relationship between a robot end-effector and skin contact is established using an impedance control model. To solve the problem that the reference trajectory is difficult to obtain, a skin mechanical model is established to estimate the environmental boundary of impedance control. To address the problem that impedance control parameters are difficult to adjust, a reinforcement learning algorithm is constructed by combining a neural network and a cross-entropy method for control parameter search. The state transition model constructed using a BP neural network can be updated offline, accelerating the search for optimal control parameters, which optimizes the problem of slow reinforcement learning convergence. The uncertainty of the contact process is considered using a probabilistic statistics-based approach to strategy search. Experimental results show that the model-based reinforcement learning algorithm for force control can obtain a relatively smooth force compared to traditional PID algorithms, and the error is basically within ± 0.2 N during the online experiment.