An Online Intelligent Visual Interaction System

This paper proposes an Online Intelligent Visual Interactive System (OIVIS), which can be applied to various short video and live streaming apps to provide an interactive user experience. During the live streaming scene, the anchor can issue various commands by using pre-defined gestures, and can trigger real-time background replacement to create an immersive atmosphere.To support such dynamic interactivity, we implemented algorithms including real-time gesture recognition and real-time video portrait segmentation, developed a lightweight deep neural network inference engine called MNN, and a real-time rendering framework AI Gender at the front-end to create a complete set of visual interaction solutions for using in resource constrained mobile.