The limitations of differentiable architecture search

被引：0

作者：

Guillaume, Lacharme ^{[1
]}

Hubert, Cardot ^{[1
]}

Christophe, Lente ^{[1
]}

Nicolas, Monmarche ^{[1
]}

机构：

[1] Univ Tours, LIFAT, 64 Ave Jean Portalis, F-37200 Tours, France

来源：

PATTERN ANALYSIS AND APPLICATIONS | 2024年 / 27卷 / 02期

关键词：

Differentiable architecture search (DARTS); Bi-level optimization; Depth regularization;

D O I：

10.1007/s10044-024-01260-5

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

In this paper, we will provide a detailed explanation of the limitations behind differentiable architecture search (DARTS). Algorithms based on the DARTS paradigm tend to converge towards degenerate solutions. A degenerate solution corresponds to an architecture with a shallow graph containing mainly skip connections. We have identified 6 sources of errors that could explain this phenomenon. Some of these errors can only be partially eliminated. Therefore, we will propose an innovative solution to remove degenerate solutions from the search space. We will demonstrate the validity of our approach through experiments conducted on the CIFAR10 and CIFAR100 databases. Our code is available at the following link: https://scm.univ-tours.fr/projetspublics/lifat/darts_ibpria_sparcity

引用

页数：14

共 50 条

[41] Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluation
Chen, Xin
Xie, Lingxi
Wu, Jun
Tian, Qi
2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, : 1294 - 1303
[42] Differentiable neural architecture search for domain adaptation in fault diagnosis
Liu, Yumeng
Li, Xudong
Hu, Yang
MECHANICAL SYSTEMS AND SIGNAL PROCESSING, 2023, 202
[43] Efficient Differentiable Architecture Search with Backbone and FPN for Object Detection
Zhang, Qiyu
Han, Hongui
Li, Fangyu
Du, Yongping
2024 14TH ASIAN CONTROL CONFERENCE, ASCC 2024, 2024, : 1908 - 1913
[44] Zero-Cost Operation Scoring in Differentiable Architecture Search
Xiang, Lichuan
Dudziak, Lukasz
Abdelfattah, Mohamed S.
Chau, Thomas
Lane, Nicholas D.
Wen, Hongkai
THIRTY-SEVENTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, VOL 37 NO 9, 2023, : 10453 - 10463
[45] Inner Loop-Based Modified Differentiable Architecture Search
Jin, Cong
Huang, Jinjie
IEEE ACCESS, 2024, 12 : 41918 - 41933
[46] DMNAS: Differentiable Multi-modal Neural Architecture Search
Funoki, Yushiro
Ono, Satoshi
INTERNATIONAL WORKSHOP ON ADVANCED IMAGING TECHNOLOGY (IWAIT) 2021, 2021, 11766
[47] AlphaGAN: Fully Differentiable Architecture Search for Generative Adversarial Networks
Tian, Yuesong
Shen, Li
Su, Guinan
Li, Zhifeng
Liu, Wei
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2022, 44 (10) : 6752 - 6766
[48] RAPDARTS: Resource-Aware Progressive Differentiable Architecture Search
Green, Sam
Vineyard, Craig M.
Helinski, Ryan
Koc, Cetin Kaya
2020 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2020,
[49] Inner Loop-Based Modified Differentiable Architecture Search
Jin, Cong
Huang, Jinjie
IEEE Access, 2024, 12 : 41918 - 41933
[50] Improving Differentiable Architecture Search via self-distillation
Zhu, Xunyu
Li, Jian
Liu, Yong
Wang, Weiping
NEURAL NETWORKS, 2023, 167 : 656 - 667

← 1 2 3 4 5 →