Resource-aware in-edge distributed real-time deep learning

Deep neural networks (DNNs) are widely used in IoT devices for applications like pattern recognition. However, slight variations in the input data may cause considerable accuracy loss, while capturing all data variations to provide a rich training dataset is almost unrealistic. Online learning can assist by offering to continue adapting the model to the data variations even during inference, however at the expense of higher resource demands, namely a challenging requirement for resource-constrained IoT devices. Furthermore, training on a data sample must be concluded in a timely manner, to have the model updated for subsequent data inferences, compelling the data inter-arrival time as a time constraint. Distributed learning can mitigate the per-device resource demand by splitting the model and placing the partitions on the IoT devices. However, the previous distributed learning studies primarily aim to improve the throughput (through accelerating the training by large-scale CPU or GPU clusters), with less attention to the timeliness constraints. This paper, however, pays attention to some application-specific constraints of timeliness and accuracy under IoT device resource limitations using modular neural networks (MNNs). The MNN clusters the input space using a proposed online approach, where a module is specialized to each of the dynamic data clusters to perform inference. The MNN adjusts its computational complexity adaptively by adding, removing, and tuning the module clusters as new data arrives. The simulation results show that the proposed method effectively adheres to the application constraints and the device resource limitations.