Model Hyper Parameter Tuning using Ant Colony Optimization
Date Issued
2022
Author(s)
Trajkovski, Aleksandar
Madjarov, Gjorgji
Abstract
The process of adjusting the best hyper-parameters
for machine learning models is a complex optimization problem
over the models that in reality are ”black-boxes”. This process
entails several challenges such as the time complexity of finding
optimal results, different types of hyper-parameters (continuous
and categorical hyper-parameters) and model over-fitting. The
main benefits of the hyper-parameter adjustment process are
the improvement of the efficiency and quality of the models.
This paper presents an implementation for adjusting hyperparameters of time series prediction regression models, which
is based on pheromone paths of ant colonies used as a way
of finding food in nature. This approach of leaving pheromone
pathways has been used to quickly find values for the hyperparameters of the regression models for which they give the
most optimal results.
The proposed implementation for Ant Colony optimization is
compared with three other parameter optimization approaches
(Grid search, Bayesian optimization and Tree Parzen Estimators) on ten different regression datasets according to the time
required to perform the optimization and the quality of the
model prediction. The experimental evaluation shows that the
proposed method needs minimal time to optimize the hyperparameters, while preserving very good predictive performance
in comparison to the competing approaches.
for machine learning models is a complex optimization problem
over the models that in reality are ”black-boxes”. This process
entails several challenges such as the time complexity of finding
optimal results, different types of hyper-parameters (continuous
and categorical hyper-parameters) and model over-fitting. The
main benefits of the hyper-parameter adjustment process are
the improvement of the efficiency and quality of the models.
This paper presents an implementation for adjusting hyperparameters of time series prediction regression models, which
is based on pheromone paths of ant colonies used as a way
of finding food in nature. This approach of leaving pheromone
pathways has been used to quickly find values for the hyperparameters of the regression models for which they give the
most optimal results.
The proposed implementation for Ant Colony optimization is
compared with three other parameter optimization approaches
(Grid search, Bayesian optimization and Tree Parzen Estimators) on ten different regression datasets according to the time
required to perform the optimization and the quality of the
model prediction. The experimental evaluation shows that the
proposed method needs minimal time to optimize the hyperparameters, while preserving very good predictive performance
in comparison to the competing approaches.
Subjects
File(s)![Thumbnail Image]()
Loading...
Name
CIIT_2022_7.pdf
Size
248.94 KB
Format
Adobe PDF
Checksum
(MD5):b3299cbeab4a3d503e37fd5e68e1bd9b