Acoustic-based Alphanumeric Input Interface for Earables

As earables gain popularity, there emerges a need for intuitive user interfaces that adapt to diverse daily scenarios. Traditional methods like touchscreens and voice control often fall short in environments like movie theatres, where silence and darkness are required, or on busy streets where visual distraction introduces extra risk. We propose an innovative earable-based system utilizing unique acoustic friction generated by fingers for alphanumeric input. Our approach digs into the acoustic friction theory, applying this knowledge to better understand the transformation from 2D handwriting into a 1D acoustic time series. This theoretical foundation guides our system design and feature extraction. Specifically, we have redesigned certain characters to enhance their acoustic distinctiveness without compromising the natural handwriting style of users, ensuring the system user-friendly. Our system combines DenseNet and GRU architectures in a multimodal model, refined through transfer learning to adapt to diverse user behaviors. Tested in real-world scenarios with 10 participants, our system achieves a 95% accuracy in recognizing both letters and numbers.