We study the multiobjective hyperparameter optimization problem that arises in classification and regression type settings in renewable energy markets. Examples of objectives include accuracy, computational time, and bias. Models can range from complex decision trees and neural networks to simpler KNNs (K-Nearest Neighbours). Multiobjective hyperparameter optimization has entailed deployment of both Bayesian and Direct-Search methods individually in the past. In this paper, we propose a switching framework that effectively uses both of these methods in an iterative fashion. As an additional contribution, we propose a warm-start based training for the machine learning models, which effectively deploys the benefits of Direct-Search methods. We compare our method with traditional Bayesian and Direct-Search methods by using multiple real-world datasets and machine learning models. We observe a clear-cut computational advantage in using our “combined method” and propose to extend this to general derivative-free regimes in the future.