Advanced Driver Assistance System based on NeuroFSM applied in the detection of autonomous human faults and support to semi-autonomous control for robotic vehicles

This paper presents an ADAS (Advanced Driver Assistance System) applied in the detection of human faults and to support the semi-autonomous control of robotic vehicles in environments subject to traffic rules. The system must be able to detect and classify several different human faults which are related to a non-compliance with the local traffic rules (e.g. maximum speed allowed, stop signal, slow down, turn right/left, prohibited direction, pedestrian crossing zone), thus helping to make navigation according to the local traffic rules. We also use a new approach termed as Neuro-FSM (Neural Finite State Machine), to assess the state of the vehicle. Our ADAS system for detecting human faults, based in the Neuro-FSM, achieved an accuracy of 92.1% in the detection and classification of human actions (correct/incorrect behavior), having a great potential for the reduction of traffic accidents. The results are promising and very satisfactory, where we also obtained 98.3% of accuracy in the sign classification task in a traffic signal benchmark dataset (INI - German Traffic Sign Benchmark) and 83% of accuracy in the task of detecting traffic signs using 3D images in a dataset from KITTI (KITTI Vision Benchmark Suite). Through the traffic sign detection and recognition system, it was possible to compare the behavior of the driver and the vehicle state (via vehicle captured data speed, steering, braking and acceleration), with the expected car navigation behavior according to the traffic rules present in the environment. Thus, allowing the detection of human car conduction failures, caused by imprudence or lack of attention to the visual signs (traffic rules).