Combined Bayesian and RNN-Based Hyperparameter Optimization for Efficient Model Selection Applied for autoML

Ruei Sing Guan*, Yu Chee Tseng, Jen Jee Chen, Po Tsun Kuo

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

The field of hyperparameter optimization (HPO) in auto machine learning (autoML) has been intensively studied, mainly in auto model selection (AMS), which finds the best set of hyperparameters, and neural architecture search (NAS), which optimizes the architecture of deep learning networks. In HPO, the two most significant problems are the demand of high level computational resources and the need of enormous computational time (GPU hours). In particular, the computational resources spent on HPO for complex deep learning networks are extremely high. Therefore, this paper augments HPO by adding recurrent neural networks (RNNs) to traditional statistical model-based algorithms to reduce the number of iterations of statistical models and eventually achieve the goal of lowering required computational resources. This paper’s main contribution is combining traditional statistical model-based algorithms and recurrent neural network models to reduce the computational time when doing HPO with deep learning.

Original languageEnglish
Title of host publicationNew Trends in Computer Technologies and Applications - 25th International Computer Symposium, ICS 2022, Proceedings
EditorsSun-Yuan Hsieh, Ling-Ju Hung, Sheng-Lung Peng, Ralf Klasing, Chia-Wei Lee
PublisherSpringer Science and Business Media Deutschland GmbH
Pages86-97
Number of pages12
ISBN (Print)9789811995811
DOIs
StatePublished - 2022
Event25th International Computer Symposium on New Trends in Computer Technologies and Applications, ICS 2022 - Taoyuan, Taiwan
Duration: 15 Dec 202217 Dec 2022

Publication series

NameCommunications in Computer and Information Science
Volume1723 CCIS
ISSN (Print)1865-0929
ISSN (Electronic)1865-0937

Conference

Conference25th International Computer Symposium on New Trends in Computer Technologies and Applications, ICS 2022
Country/TerritoryTaiwan
CityTaoyuan
Period15/12/2217/12/22

Keywords

  • autoML
  • Bayesian Optimization (BO)
  • Hyperparameter Optimization (HPO)
  • Recurrent Neural Network (RNN)

Fingerprint

Dive into the research topics of 'Combined Bayesian and RNN-Based Hyperparameter Optimization for Efficient Model Selection Applied for autoML'. Together they form a unique fingerprint.

Cite this