Touch Classification on Robotic Skin using Multimodal Tactile Sensing Modules

Human employs different touch patterns to convey diverse social messages; for example, a stroke is an encouragement, whereas a hit is an offense. Various tactile sensors have been developed to grant an intuitive physical interaction with a robotic system, yet many encountered limitations in achieving broad sensibility or fabricating into a large skin. This paper presents a robotic skin with multimodal tactile sensing modules to achieve broad spatiotemporal sensibility with a few sensing elements. The multimodal module is composed of a microphone and a vented screw installed on a conductive sensory domain. A multilayered fabric with a textured surface covers the sensory domain and forms a piezoresistive structure. High and low temporal components of touch elicit a micro-vibration and a conductivity change on the skin, where both are measured with multimodal modules. The measurements are each processed with short-time Fourier transform (STFT) and electrical resistance tomography (ERT) to encode two spatiotemporal feature maps, which are classified into ten touch classes using a convolutional neural network. Due to a sensibility to both high and low temporal components of touch, the skin classifies touches with an accuracy of 97.0 %, whereas only 84.7 % and 90.6 % are achieved when one type of feature map is used. Also, the skin is robust and beneficial in power consumption and fabrication since the multimodal modules are not exposed to an external stimulus and are sparsely distributed.