Perception simplex: Verifiable collision avoidance in autonomous vehicles amidst obstacle detection faults

Advances in deep learning have revolutionized cyber-physical applications, including the development of autonomous vehicles. However, real-world collisions involving autonomous control of vehicles have raised significant safety concerns regarding the use of deep neural networks (DNNs) in safety-critical tasks, particularly perception. The inherent unverifiability of DNNs poses a key challenge in ensuring their safe and reliable operation. In this work, we propose perception simplex (PS$$ \mathcal{PS} $$), a fault-tolerant application architecture designed for obstacle detection and collision avoidance. We analyse an existing LiDAR-based classical obstacle detection algorithm to establish strict bounds on its capabilities and limitations. Such analysis and verification have not been possible for deep learning-based perception systems yet. By employing verifiable obstacle detection algorithms, PS$$ \mathcal{PS} $$ identifies obstacle existence detection faults in the output of unverifiable DNN-based object detectors. When faults with potential collision risks are detected, appropriate corrective actions are initiated. Through extensive analysis and software-in-the-loop simulations, we demonstrate that PS$$ \mathcal{PS} $$ provides deterministic fault tolerance against obstacle existence detection faults, establishing a robust safety guarantee. The traditional autonomous driving system (mission layer) is monitored at runtime by the safety layer, which includes verifiable algorithms only. Utilizing the detectability model for such perception algorithms, safety layer provides deterministic fault detection and collision avoidance properties. This framework presents a promising approach towards the end goal of verifiable end-to-end safety in autonomous vehicles. Detectability model provides deterministic translation between safety policies, sensors and algorithm parameters and safety guarantees. Such a model can only be devised for fully analysable and verifiable solutions, a requirement not met by deep neural networks. image