TY - GEN
T1 - Combined Bayesian and RNN-Based Hyperparameter Optimization for Efficient Model Selection Applied for autoML
AU - Guan, Ruei Sing
AU - Tseng, Yu Chee
AU - Chen, Jen Jee
AU - Kuo, Po Tsun
N1 - Publisher Copyright:
© 2022, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
PY - 2022
Y1 - 2022
N2 - The field of hyperparameter optimization (HPO) in auto machine learning (autoML) has been intensively studied, mainly in auto model selection (AMS), which finds the best set of hyperparameters, and neural architecture search (NAS), which optimizes the architecture of deep learning networks. In HPO, the two most significant problems are the demand of high level computational resources and the need of enormous computational time (GPU hours). In particular, the computational resources spent on HPO for complex deep learning networks are extremely high. Therefore, this paper augments HPO by adding recurrent neural networks (RNNs) to traditional statistical model-based algorithms to reduce the number of iterations of statistical models and eventually achieve the goal of lowering required computational resources. This paper’s main contribution is combining traditional statistical model-based algorithms and recurrent neural network models to reduce the computational time when doing HPO with deep learning.
AB - The field of hyperparameter optimization (HPO) in auto machine learning (autoML) has been intensively studied, mainly in auto model selection (AMS), which finds the best set of hyperparameters, and neural architecture search (NAS), which optimizes the architecture of deep learning networks. In HPO, the two most significant problems are the demand of high level computational resources and the need of enormous computational time (GPU hours). In particular, the computational resources spent on HPO for complex deep learning networks are extremely high. Therefore, this paper augments HPO by adding recurrent neural networks (RNNs) to traditional statistical model-based algorithms to reduce the number of iterations of statistical models and eventually achieve the goal of lowering required computational resources. This paper’s main contribution is combining traditional statistical model-based algorithms and recurrent neural network models to reduce the computational time when doing HPO with deep learning.
KW - Bayesian Optimization (BO)
KW - Hyperparameter Optimization (HPO)
KW - Recurrent Neural Network (RNN)
KW - autoML
UR - http://www.scopus.com/inward/record.url?scp=85150994185&partnerID=8YFLogxK
U2 - 10.1007/978-981-19-9582-8_8
DO - 10.1007/978-981-19-9582-8_8
M3 - Conference contribution
AN - SCOPUS:85150994185
SN - 9789811995811
T3 - Communications in Computer and Information Science
SP - 86
EP - 97
BT - New Trends in Computer Technologies and Applications - 25th International Computer Symposium, ICS 2022, Proceedings
A2 - Hsieh, Sun-Yuan
A2 - Hung, Ling-Ju
A2 - Peng, Sheng-Lung
A2 - Klasing, Ralf
A2 - Lee, Chia-Wei
PB - Springer Science and Business Media Deutschland GmbH
T2 - 25th International Computer Symposium on New Trends in Computer Technologies and Applications, ICS 2022
Y2 - 15 December 2022 through 17 December 2022
ER -