Video-based fusion of multiple detectors to counter terrorism

Terrorism is an international security challenge. The early detection of threats (e.g., explosives or firearms) could provide a valuable contribution to the ability to prevent, protect and respond to terrorism. This paper presents a system for the management of a plurality of sensors to improve the threat-detection capabilities without disrupting the flow of passengers. The system improves the prevention capabilities of soft targets (such as airports, undergrounds and railway stations) with a high number of daily commuters. The system architecture consists of three main components. The first component is 2D video tracking and re-identification (Re-ID), which allows the labelling and tracking of commuters in a small area. Thereby, it supports the fusion of sensors at different locations. The Re-ID has a smart training strategy with anonymized snippets to increase flexibility for new environments. The second component is 3D video tracking with a stereo camera, which gives a more accurate location measurement than 2D video. Location prediction is used to compensate for latency in the control of active elements in the threat detection sensor. Recurrent neural networks for location prediction were trained by using real 3D tracking data from a railway station. The performance is evaluated with a ground-truth based on Ultra-Wide Band (UWB) radio positioning and a coordinate conversion method was created to compensate for identified inaccuracies. The third component is Command & Control (C&C), which consists of three submodules: message broker, data-fusion and security client. The message broker is a publish-subscribe middleware layer to enable flexible integration of the various sensors and components. The data-fusion combines outputs of multiple sensors. In case of a suspect person, the security client triggers an alarm and a comprehensive report is sent to the security guards.