Efficient Deep Structure Learning for Resource-Limited IoT Devices

被引:2
|
作者
Shen, Shibo [1 ]
Li, Rongpeng [1 ]
Zhao, Zhifeng [2 ]
Liu, Qing [3 ]
Liang, Jing [3 ]
Zhang, Honggang [1 ]
机构
[1] Zhejiang Univ, Coll Informat Sci & Elect Engn, Hangzhou, Peoples R China
[2] Zhejiang Lab, Hangzhou, Peoples R China
[3] Huawei Technol Co Ltd, Shanghai, Peoples R China
来源
2020 IEEE GLOBAL COMMUNICATIONS CONFERENCE (GLOBECOM) | 2020年
基金
中国国家自然科学基金; 国家重点研发计划;
关键词
deep neural networks; deep learning; Internet of Things; resource-limited edge computing; pruning; efficiency; INTERNET; THINGS;
D O I
10.1109/GLOBECOM42002.2020.9322206
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Nowadays, deep neural networks (DNNs) have been rapidly deployed to realize a number of functionalifies like sensing, imaging, classification, recognition, etc. However. the computational-intensive requirement of DNNs makes it difficult to be applicable for resource-limited Internet of Things (loT) devices. In this paper, we propose a novel pruning-based paradigm that aims to reduce the computational cost of DNNs. by uncovering a more compact structure and learning the effective weights therein, on the basis of not compromising the expressive capability of DNNs. In particular, our algorithm can achieve efficient end-to-end training that transfers a redundant neural network to a compact one with a specifically targeted compression rale directly. We comprehensively evaluate our approach on various representative benchmark datasets and compared with typical advanced convolutional neural network (C:NN) architectures. The experimental results verify the superior performance and robust effectiveness of our scheme. For example, when pruning VGG on CIFAR-10, our proposed scheme is able to significantly reduce its FLOPs (floating-point operations) and number of parameters with a proportion of 76.2% and 94.1%, respectively, while still maintaining a satisfactory accuracy. To sum up, our scheme could facilitate the integration of DNNs into the common machine-learning-based loT framework, and establish distributed training of neural networks in both cloud and edge.
引用
收藏
页数:6
相关论文
共 50 条
  • [21] Efficient stereo vision algorithms for resource-limited systems
    Tippetts, Beau
    Lee, Dah Jye
    Lillywhite, Kirt
    Archibald, James
    JOURNAL OF REAL-TIME IMAGE PROCESSING, 2015, 10 (01) : 163 - 174
  • [22] Efficient stereo vision algorithms for resource-limited systems
    Beau Tippetts
    Dah Jye Lee
    Kirt Lillywhite
    James Archibald
    Journal of Real-Time Image Processing, 2015, 10 : 163 - 174
  • [23] Efficient Detection in Aerial Images for Resource-Limited Satellites
    Li, Leyang
    Cao, Guixing
    Liu, Jun
    Tong, Yu
    IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, 2022, 19
  • [24] A Lightweight Concept Drift Detection Method for On-Device Learning on Resource-Limited Edge Devices
    Yamada, Takeya
    Matsutani, Hiroki
    2023 IEEE INTERNATIONAL PARALLEL AND DISTRIBUTED PROCESSING SYMPOSIUM WORKSHOPS, IPDPSW, 2023, : 761 - 768
  • [25] Soap and corba productivity comparison for resource-limited mobile devices
    Gryazin, EA
    Eng, ME
    Seppala, O
    Proceedings of the IASTED International Conference on Software Engineering, 2004, : 707 - 712
  • [26] Quality of medical devices and in vitro diagnostics in resource-limited settings
    Mori, Marcella
    Ravinetto, Raffaella
    Jacobs, Jan
    TROPICAL MEDICINE & INTERNATIONAL HEALTH, 2011, 16 (11) : 1439 - 1449
  • [27] Memory-aware Efficient Deep Learning Mechanism for IoT Devices
    Banerjee, Jishnu
    Islam, Sahidul
    Wei, Wei
    Pan, Chen
    Zhu, Dakai
    Xie, Mimi
    2021 IEEE 32ND INTERNATIONAL CONFERENCE ON APPLICATION-SPECIFIC SYSTEMS, ARCHITECTURES AND PROCESSORS (ASAP 2021), 2021, : 187 - 194
  • [28] Energy-Efficient Scheduling and Resource Allocation for Power-limited Cognitive IoT Devices
    Wang, Kang
    Wu, Peiran
    Xia, Minghua
    2023 19TH INTERNATIONAL CONFERENCE ON WIRELESS AND MOBILE COMPUTING, NETWORKING AND COMMUNICATIONS, WIMOB, 2023, : 104 - 109
  • [29] Enabling Deep Learning on IoT Devices
    Tang, Jie
    Sun, Dawei
    Liu, Shaoshan
    Gaudiot, Jean-Luc
    COMPUTER, 2017, 50 (10) : 92 - 96
  • [30] Efficient Embedded Learning for IoT Devices
    Venkataramani, Swagath
    Roy, Kaushik
    Raghunathan, Anand
    2016 21ST ASIA AND SOUTH PACIFIC DESIGN AUTOMATION CONFERENCE (ASP-DAC), 2016, : 308 - 311