Semi-Automatic Validation and Verification Framework for CV&AI-Enhanced Railway Signaling and Landmark Detector

The automation of railway operations is an activity in constant growth. Different railway stakeholders are already developing their research activities for the future driverless autonomous driving based on computer vision (CV) and artificial intelligence (AI)-enhanced perception technologies (e.g., obstacle detection). Unfortunately, the AI models are opaque in nature, and there are no certification accepted rules for CV & AI-enhanced functionality certification. Capturing and labeling camera image in real environment is expensive in terms of time and resources and it does not guarantee enough variation in edge visibility conditions, which makes the resulting database less valuable for the validation and verification (V & V) processes. To meet the increasing needs of trusted CV & AI-based solutions, numerous V & V approaches have been proposed in other sectors such as automotive, most of them based on virtual simulators. Unfortunately, there is currently no virtual perception simulator for railway scenario. This work aims to create a semi-automatic system based on virtual scenarios measuring the CV & AI-enhanced system performance facing different visibility conditions. It will be based on the global accuracy metrics and detected potential safety and operation rules' violations. This work also demonstrates the quantitative and qualitative improvements while reducing current V & V cost.