The limitations of differentiable architecture search

被引：0
|
作者：
Guillaume, Lacharme [1 ]
Hubert, Cardot [1 ]
Christophe, Lente [1 ]
Nicolas, Monmarche [1 ]
机构：
[1] Univ Tours, LIFAT, 64 Ave Jean Portalis, F-37200 Tours, France
来源：
PATTERN ANALYSIS AND APPLICATIONS | 2024年 / 27卷 / 02期
关键词：
Differentiable architecture search (DARTS); Bi-level optimization; Depth regularization;
D O I：
10.1007/s10044-024-01260-5
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
In this paper, we will provide a detailed explanation of the limitations behind differentiable architecture search (DARTS). Algorithms based on the DARTS paradigm tend to converge towards degenerate solutions. A degenerate solution corresponds to an architecture with a shallow graph containing mainly skip connections. We have identified 6 sources of errors that could explain this phenomenon. Some of these errors can only be partially eliminated. Therefore, we will propose an innovative solution to remove degenerate solutions from the search space. We will demonstrate the validity of our approach through experiments conducted on the CIFAR10 and CIFAR100 databases. Our code is available at the following link: https://scm.univ-tours.fr/projetspublics/lifat/darts_ibpria_sparcity
引用
收藏
页数：14
相关论文
共 50 条
  • [31] Operation-level Progressive Differentiable Architecture Search
    Zhu, Xunyu
    Li, Jian
    Liu, Yong
    Liao, Jun
    Wang, Weiping
    2021 21ST IEEE INTERNATIONAL CONFERENCE ON DATA MINING (ICDM 2021), 2021, : 1559 - 1564
  • [32] DropNAS: Grouped Operation Dropout for Differentiable Architecture Search
    Hong, Weijun
    Li, Guilin
    Zhang, Weinan
    Tang, Ruiming
    Wang, Yunhe
    Li, Zhenguo
    Yu, Yong
    PROCEEDINGS OF THE TWENTY-NINTH INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE, 2020, : 2326 - 2332
  • [33] DOTS: Decoupling Operation and Topology in Differentiable Architecture Search
    Gu, Yu-Chao
    Wang, Li-Juan
    Liu, Yun
    Yang, Yi
    Wu, Yu-Huan
    Lu, Shao-Ping
    Cheng, Ming-Ming
    2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, : 12306 - 12315
  • [34] Operation and Topology Aware Fast Differentiable Architecture Search
    Siddiqui, Shahid
    Kyrkou, Christos
    Theocharides, Theocharis
    2020 25TH INTERNATIONAL CONFERENCE ON PATTERN RECOGNITION (ICPR), 2021, : 9666 - 9673
  • [35] iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradients
    Zhang, Miao
    Su, Steven
    Pan, Shirui
    Chang, Xiaojun
    Abbasnejad, Ehsan
    Haffari, Reza
    INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 139, 2021, 139
  • [36] Understanding the wiring evolution in differentiable neural architecture search
    Xie, Sirui
    Hu, Shoukang
    Wang, Xinjiang
    Liu, Chunxiao
    Shi, Jianping
    Liu, Xunying
    Lin, Dahua
    24TH INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE AND STATISTICS (AISTATS), 2021, 130
  • [37] Differentiable Architecture Search Algorithm Based on Global Comparison
    Zeng, Xianglun
    Xiao, Hongxiang
    IEEE ACCESS, 2023, 11 : 82674 - 82684
  • [38] Memory-Efficient Differentiable Transformer Architecture Search
    Zhao, Yuekai
    Dong, Li
    Shen, Yelong
    Zhang, Zhihua
    Wei, Furu
    Chen, Weizhu
    FINDINGS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, ACL-IJCNLP 2021, 2021, : 4254 - 4264
  • [39] Exploiting Operation Importance for Differentiable Neural Architecture Search
    Zhou, Yuan
    Xie, Xukai
    Kung, Sun-Yuan
    IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2022, 33 (11) : 6235 - 6248
  • [40] Adaptive Channel Allocation for Robust Differentiable Architecture Search
    Li, Chao
    Ning, Jia
    Hu, Han
    He, Kun
    IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2024,
12345