Speeding up Common Hyperparameter Optimization Methods by a Two-Phase-Search

被引：0

作者：

Wendt, Alexander ^{[1
]}

Wuschnig, Marco ^{[1
]}

Lechner, Martin ^{[1
]}

机构：

[1] TU Vienna, ICT, Christian Doppler Lab Embedded Machine Learning, Vienna, Austria

来源：

IECON 2020: THE 46TH ANNUAL CONFERENCE OF THE IEEE INDUSTRIAL ELECTRONICS SOCIETY | 2020年

关键词：

hyperparameter; machine learning; support vector machine; random forest; Bayesian optimization; optimization;

D O I：

10.1109/iecon43393.2020.9254801

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

Hyperparameter search concerns everybody who works with machine learning. We compare publicly available hyperparameter searches on four datasets. We develop metrics to measure the performance of hyperparameter searches across datasets of different sizes as well as machine learning algorithms. Further, we propose a method of speeding up the search by using subsets of data. Results show that random search performs well compared to Bayesian methods and that a combined search can speed up the search by a factor of 5.

引用

页码：517 / 522

页数：6

共 44 条

[1] Speeding up the Hyperparameter Optimization of Deep Convolutional Neural Networks
Hinz, Tobias
Navarro-Guerrero, Nicolas
Magg, Sven
Wermter, Stefan
INTERNATIONAL JOURNAL OF COMPUTATIONAL INTELLIGENCE AND APPLICATIONS, 2018, 17 (02)
[2] Speeding up dynamic search methods in speech recognition
Gosztolya, G
Kocsor, A
INNOVATIONS IN APPLIED ARTIFICIAL INTELLIGENCE, 2005, 3533 : 98 - 100
[3] Speeding Up Automatic Hyperparameter Optimization of Deep Neural Networks by Extrapolation of Learning Curves
Domhan, Tobias
Springenberg, Jost Tobias
Hutter, Frank
PROCEEDINGS OF THE TWENTY-FOURTH INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE (IJCAI), 2015, : 3460 - 3468
[4] TransBO: Hyperparameter Optimization via Two-Phase Transfer Learning
Li, Yang
Shen, Yu
Jiang, Huaijun
Zhang, Wentao
Yang, Zhi
Zhang, Ce
Cui, Bin
PROCEEDINGS OF THE 28TH ACM SIGKDD CONFERENCE ON KNOWLEDGE DISCOVERY AND DATA MINING, KDD 2022, 2022, : 956 - 966
[5] Speeding Up Chemical Searches Using the Inverted Index: The Convergence of Chemoinformatics and Text Search Methods
Nasr, Ramzi
Vernica, Rares
Li, Chen
Baldi, Pierre
JOURNAL OF CHEMICAL INFORMATION AND MODELING, 2012, 52 (04) : 891 - 900
[6] Data structures for speeding up Tabu Search when solving sparse quadratic unconstrained binary optimization problems
Liang, Ricardo N.
Anacleto, Eduardo A. J.
Meneses, Claudio N.
JOURNAL OF HEURISTICS, 2022, 28 (04) : 433 - 479
[7] Data structures for speeding up Tabu Search when solving sparse quadratic unconstrained binary optimization problems
Ricardo N. Liang
Eduardo A. J. Anacleto
Cláudio N. Meneses
Journal of Heuristics, 2022, 28 : 433 - 479
[8] Speeding Up Multi-Objective Hyperparameter Optimization by Task Similarity-Based Meta-Learning for the Tree-Structured Parzen Estimator
Watanabe, Shuhei
Awad, Noor
Onishi, Masaki
Hutter, Frank
PROCEEDINGS OF THE THIRTY-SECOND INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE, IJCAI 2023, 2023, : 4380 - 4388
[9] Speeding up the two-phase PT-flash, with applications for calculation of miscible displacement
Michael, ML
FLUID PHASE EQUILIBRIA, 1998, 143 (1-2) : 1 - 12
[10] Two-step spectral library pre-search: A novel approach for speeding up compound identification
Zhu, Qiang
Yu, Jiancheng
Hu, Junjack
Ding, Chuanfan
INTERNATIONAL JOURNAL OF MASS SPECTROMETRY, 2017, 417 : 40 - 44

← 1 2 3 4 5 →