Our paper’s focus is on addressing model selection in a multi-model context, where hyperparameter optimization is also involved. To resolve the problem, our proposal involves a two-tiered approach. The first tier involves a multi-armed Gaussian Bandit algorithm that selects the model, while the second tier uses a Gaussian process-based Bayesian optimization technique to determine the optimal hyperparameters. Our method outperforms random search and improves upon previous work by expanding model selection to include both hyperparameter and model family selection. We provide a thorough description of our system and discuss potential directions for further improving automated model selection systems. The findings from our results show that Thompson Sampling is more likely to explore multiple model types than Approximate Value Function Lookahead.