Design possibilities and challenges of DNN models: a review on the perspective of end devices

Deep Neural Network (DNN) models for both resource-rich environments and resource-constrained devices have become abundant in recent years. As of now, the literature on different available options for the design, development, and deployment of DNN models to resource constrained-end devices is limited and demands extensive further study. This paper reviews vital research efforts for the design of DNN models while deploying them at the end devices such as smart cameras for real-time object detection tasks. The design ideas include the types of DNN models, hardware and software requirements for the development, resource constraints imposed by the computing devices, and the optimization techniques required for the efficient processing of DNN. The study also aims to conduct a systematic literature review on current trends in different real-time applications of DNN models and explores the following four dimensions: (1) DNN model perspective: to associate appropriate DNN models with the proper hardware to achieve optimal throughput. (2) Hardware perspective: to answer different available options in hardware platforms for achieving on-device intelligence. (3) Resources and optimization perspective: to analyze the type of resource limitations in hardware platforms and the use of optimization techniques to overcome the performance issues. (4) Application perspective: to understand the real-time uses of DNN models in different application domains. This work also explores different performance measures that need to be considered for on-device intelligence and provides possible future directions for the challenges reviewed.