Extraction of visual and acoustic features of the driver for monitoring driver ergonomics applied to extended driver assistance systems

The National Highway Traffic Safety Administration (NHTSA) estimates that in the USA alone approximately 100,000 crashes each year are caused primarily by driver drowsiness or fatigue. The major cause for inattentiveness has been found to be a deficit in what we call in this paper an extended view of ergonomics, i.e. the "extended ergonomics status" of the driving process. This deficit is multidimensional as it includes aspects such as drowsiness (sleepiness), fatigue (lack of energy) and emotions/stress (for example sadness, anger, joy, pleasure, despair and irritation). Different approaches have been proposed for monitoring driver states, especially drowsiness and fatigue, using visual features of the driver such as head movement patterns eyelid movements, facial expressions or all of these together. The effectiveness of the approach depends on the quality of the extracted features, efficiency and the responsiveness of the classification algorithm. In this work, we propose the usage of acoustic information along with visual features to increase the robustness of the emotion/stress measurement system. In terms of the acoustic signals, this work will enlist the appropriate features for the driving situation and correlate them to parameters/dimensions of the "extended ergonomics status" vector. Prosodic features as well as the phonetic features of the acoustic signal are taken into account for the emotion recognition here. In this paper, a linear discriminant analysis based on a classification method using the Hausdorff distance measure is proposed for classifying the different emotional states. Experimental evaluation based on the Berlin voice database shows that the proposed method results in 85% recognition accuracy in speaker-independent emotion recognition experiments. © 2010 Springer-Verlag Berlin Heidelberg.