Mixed precision quantization of silicon optical neural network chip

被引:1
|
作者
Zhang, Ye [1 ]
Wang, Ruiting [2 ,3 ]
Zhang, Yejin [2 ,3 ]
Pan, Jiaoqing [2 ,3 ]
机构
[1] Beijing Informat Sci & Technol Univ, Beijing 100192, Peoples R China
[2] Chinese Acad Sci, Key Lab Semicond Mat Sci, Inst Semicond, Beijing 100083, Peoples R China
[3] Univ Chinese Acad Sci, Ctr Mat Sci & Optoelect Engn, Beijing 100049, Peoples R China
基金
北京市自然科学基金; 中国国家自然科学基金;
关键词
GENETIC ALGORITHM; END;
D O I
10.1016/j.optcom.2024.131231
中图分类号
O43 [光学];
学科分类号
070207 ; 0803 ;
摘要
In recent years, the field of neural network research has witnessed remarkable advancements in various domains. One of the emerging approaches is the integration of photonic computing, which leverages the unique properties of light for ultra-fast information processing. In this article, we establish a mixed precision quantization model to silicon-based optical neural networks and evaluates their performance on the MNIST and Fashion-MNIST datasets. Through a genetic algorithm- based optimization process, we achieve significant parameter compression while maintaining competitive accuracy. Our findings demonstrate that with an average quantization bitwidth of 4.5 bits on the MNIST dataset, we achieve an impressive 85.94% reduction in parameter size compared to traditional 32-bit networks, with only a marginal accuracy drop of 0.65%. Similarly, on the Fashion-MNIST dataset, we achieve an average quantization bitwidth of 5.67 bits, resulting in an 82.28% reduction in parameter size with a slight accuracy drop of 0.8%.
引用
收藏
页数:9
相关论文
共 50 条
  • [1] Mixed-precision Deep Neural Network Quantization With Multiple Compression Rates
    Wang, Xuanda
    Fei, Wen
    Dai, Wenrui
    Li, Chenglin
    Zou, Junni
    Xiong, Hongkai
    2023 DATA COMPRESSION CONFERENCE, DCC, 2023, : 371 - 371
  • [2] EVOLUTIONARY QUANTIZATION OF NEURAL NETWORKS WITH MIXED-PRECISION
    Liu, Zhenhua
    Zhang, Xinfeng
    Wang, Shanshe
    Ma, Siwei
    Gao, Wen
    2021 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP 2021), 2021, : 2785 - 2789
  • [3] Mixed Precision Low-Bit Quantization of Neural Network Language Models for Speech Recognition
    Xu, Junhao
    Yu, Jianwei
    Hu, Shoukang
    Liu, Xunying
    Meng, Helen
    IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING, 2021, 29 : 3679 - 3693
  • [4] Mixed-Precision Neural Network Quantization via Learned Layer-Wise Importance
    Tang, Chen
    Ouyang, Kai
    Wang, Zhi
    Zhu, Yifei
    Ji, Wen
    Wang, Yaowei
    Zhu, Wenwu
    COMPUTER VISION, ECCV 2022, PT XI, 2022, 13671 : 259 - 275
  • [5] DPQ: dynamic pseudo-mean mixed-precision quantization for pruned neural network
    Pei, Songwen
    Wang, Jiyao
    Zhang, Bingxue
    Qin, Wei
    Xue, Hai
    Ye, Xiaochun
    Chen, Mingsong
    MACHINE LEARNING, 2024, 113 (07) : 4099 - 4112
  • [6] Silicon-based Optical Neural Network Chip Based on Coherent Detection
    Wang, Ruiting
    Wang, Pengfei
    Luo, Guangzhen
    Yu, Hongyan
    Zhou, Xuliang
    Zhang, Yejin
    Pan, Jiaoqing
    2020 ASIA COMMUNICATIONS AND PHOTONICS CONFERENCE (ACP) AND INTERNATIONAL CONFERENCE ON INFORMATION PHOTONICS AND OPTICAL COMMUNICATIONS (IPOC), 2020,
  • [7] HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision
    Dong, Zhen
    Yao, Zhewei
    Gholami, Amir
    Mahoney, Michael W.
    Keutzer, Kurt
    2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, : 293 - 302
  • [8] CMQ: Crossbar-Aware Neural Network Mixed-Precision Quantization via Differentiable Architecture Search
    Peng, Jie
    Liu, Haijun
    Zhao, Zhongjin
    Li, Zhiwei
    Liu, Sen
    Li, Qingjiang
    IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS, 2022, 41 (11) : 4124 - 4133
  • [9] AutoMPQ: Automatic Mixed-Precision Neural Network Search via Few-Shot Quantization Adapter
    Xu, Ke
    Shao, Xiangyang
    Tian, Ye
    Yang, Shangshang
    Zhang, Xingyi
    IEEE TRANSACTIONS ON EMERGING TOPICS IN COMPUTATIONAL INTELLIGENCE, 2024, : 1 - 13
  • [10] Mixed-Precision Network Quantization for Infrared Small Target Segmentation
    Li, Boyang
    Wang, Longguang
    Wang, Yingqian
    Wu, Tianhao
    Lin, Zaiping
    Li, Miao
    An, Wei
    Guo, Yulan
    IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2024, 62 : 1 - 12