NeurIPS 2017 Workshop on Bayesian Optimization

Learning to Transfer Initializations for Bayesian Hyperparameter Optimization

Jungtaek Kim, Saehoon Kim, Seungjin Choi

We propose a neural network to learn meta-features over datasets, which is used to select initial points for Bayesian hyperparameter optimization. Specifically, we retrieve k-nearest datasets to transfer a prior knowledge on initial points, where similarity over datasets is computed by learned meta-features. Experiments demonstrate that our learned meta-features are useful in optimizing several hyperparameters of deep residual networks for image classification.