Using Pre-trained Full-Precision Models to Speed Up Training Binary Networks For Mobile Devices

被引：1

作者：

Alizadeh, Milad ^{[1
]}

Lane, Nicholas D. ^{[1
,2
]}

机构：

[1] Univ Oxford, Oxford, England

[2] Nokia Bell Labs, Murray Hill, NJ USA

来源：

MOBISYS'18: PROCEEDINGS OF THE 16TH ACM INTERNATIONAL CONFERENCE ON MOBILE SYSTEMS, APPLICATIONS, AND SERVICES | 2018年

关键词：

D O I：

10.1145/3210240.3210821

中图分类号：

TP301 [理论、方法];

学科分类号：

081202 ;

摘要：

Binary Neural Networks (BNNs) are well-suited for deploying Deep Neural Networks (DNNs) to small embedded devices but state-of-the-art BNNs need to be trained from scratch for a long time. We show how weights from a pre-trained full-precision model can be used to speed-up training of binary networks. We show that for CIFAR-10, accuracies within 1% of the full-precision model can be achieved in just 5 epochs.

引用

页码：528 / 528

页数：1

共 20 条

[1] Efficient Aspect Object Models Using Pre-trained Convolutional Neural Networks
Wilkinson, Eric
Takahashi, Takeshi
2015 IEEE-RAS 15TH INTERNATIONAL CONFERENCE ON HUMANOID ROBOTS (HUMANOIDS), 2015, : 284 - 289
[2] Training Compact Models for Low Resource Entity Tagging using Pre-trained Language Models
Izsak, Peter
Guskin, Shira
Wasserblat, Moshe
FIFTH WORKSHOP ON ENERGY EFFICIENT MACHINE LEARNING AND COGNITIVE COMPUTING - NEURIPS EDITION (EMC2-NIPS 2019), 2019, : 44 - 47
[3] Binary Classification of Skin Cancer Images Using Pre-trained Networks with I-GWO
Hussein, Hadeer
Magdy, Ahmed
Abdel-Kader, Rehab F.
Abd El Salam, Khaled
INTELIGENCIA ARTIFICIAL-IBEROAMERICAN JOURNAL OF ARTIFICIAL INTELLIGENCE, 2024, 27 (74): : 102 - 116
[4] Convolutional Neural Networks for Histopathology Image Classification: Training vs. Using Pre-Trained Networks
Kieffer, Brady
Babaie, Morteza
Kalra, Shivam
Tizhoosh, H. R.
PROCEEDINGS OF THE 2017 SEVENTH INTERNATIONAL CONFERENCE ON IMAGE PROCESSING THEORY, TOOLS AND APPLICATIONS (IPTA 2017), 2017,
[5] Graph-Based Audio Classification Using Pre-Trained Models and Graph Neural Networks
Castro-Ospina, Andres Eduardo
Solarte-Sanchez, Miguel Angel
Vega-Escobar, Laura Stella
Isaza, Claudia
Martinez-Vargas, Juan David
SENSORS, 2024, 24 (07)
[6] Training-Free Video Temporal Grounding Using Large-Scale Pre-trained Models
Zheng, Minghang
Cai, Xinhao
Chen, Qingchao
Peng, Yuxin
Liu, Yang
COMPUTER VISION-ECCV 2024, PT LXXXII, 2025, 15140 : 20 - 37
[7] Optimized classification of dental implants using convolutional neural networks and pre-trained models with preprocessed data
Reza Ahmadi Lashaki
Zahra Raeisi
Nasim Razavi
Mehdi Goodarzi
Hossein Najafzadeh
BMC Oral Health, 25 (1)
[8] What does the language system look like in pre-trained language models? A study using complex networks
Zheng, Jianyu
KNOWLEDGE-BASED SYSTEMS, 2024, 299
[9] Incident detection and classification in renewable energy news using pre-trained language models on deep neural networks
Wang, Qiqing
Li, Cunbin
JOURNAL OF COMPUTATIONAL METHODS IN SCIENCES AND ENGINEERING, 2022, 22 (01) : 57 - 76
[10] ENHANCING SEMANTIC WEB ENTITY MATCHING PROCESS USING TRANSFORMER NEURAL NETWORKS AND PRE-TRAINED LANGUAGE MODELS
Jabrane, Mourad
Toulaoui, Abdelfattah
Hafidi, Imad
COMPUTING AND INFORMATICS, 2024, 43 (06) : 1397 - 1415

← 1 2 →