Side-Channel Analysis (SCA) is critical in evaluating the security of cryptographic implementations. In recent years, the use of Deep Neural Networks (DNNs) in SCA has risen in popularity. However, DNNs consists of many hyperparameters and not every configuration of hyperparameters result in successful attack. Therefore, the search for DNN’s hyperparameters poses a significant challenge, especially when resources are limited. In this work, we explore the efficacy of a multifidelity optimization technique known as Bayesian Optimization HyperBand (BOHB) in SCA. This introduces the notion of budget within SCA to encapsulate the idea of resources when tuning the hyperparmeters of the DNNs. Next, we proposed a new objective function called \(ge_{+ntge}\) , which could be incorporated into any Bayesian Optimization used in SCA. We show the capabilities of both BOHB and \(ge_{+ntge}\) on four different public datasets. Specifically, BOHB could obtain the least number of traces in the dataset called CTF2018 when trained in the Hamming weight and identity leakage models. Notably, this marks the first reported successful recovery of the key for the identity leakage model in CTF2018.