Multi-sensor driver monitoring for drowsiness prediction

Objective Driver monitoring systems are growing in importance as well as capability. This paper reports drowsy driving detection models that use vehicular, behavioral, and physiological data. The objectives were to augment camera-based system with vehicle-based and heart rate variability measures from a wearable device and compare the performance of drowsiness detection models that use these data sources. Timeliness of the models in predicting drowsiness is analyzed. Timeliness refers to how quickly a model can identify drowsiness and, by extension, how far in advance of an adverse event a classification can be given. Methods Behavioral data were provided by a production-type Driver Monitoring System manufactured by Aisin Technical Center of America. Vehicular data were recorded from the National Advanced Driving Simulator's large-excursion motion-base driving simulator. Physiological data were collected from an Empatica E4 wristband. Forty participants drove the simulator for up to three hours after being awake for at least 16 hours. Periodic measurements of drowsiness were recorded every ten minutes using both observational rating of drowsiness by an external rater and the self-reported Karolinska Sleepiness Scale. Nine binary random forest models were created, using different combinations of data sources and ground truths. Results The classification accuracy of the nine models ranged from 0.77 to 0.92 on a scale from 0 to 1, with 1 indicating a perfect model. The best-performing model included physiological data and used a reduced dataset that eliminated missing data segments after heartrate variability measures were computed. The most timely model was able to detect the presence of drowsiness 6.7 minutes before a drowsy lane departure. Conclusions The addition of physiological measures added a small amount of accuracy to the model performance. Models trained on observational ratings of drowsiness detected drowsiness earlier than those based only on Karolinska Sleepiness Scale, making them more timely in detecting the onset of drowsiness.