Imbalance in binary classification problems refers to situations where one class (minority class) has very less samples in the dataset as compared to the other (majority class). In most real-life applications, the minority class is of high importance and hence the classifier is expected to consider this aspect for better performance of the minority class. This poses a challenge in using regular metrics like accuracy due to its inability to account for the importance in imbalanced classes. Measures like precision and recall try to overcome this drawback of accuracy by concentrating on the performance of the minority class. In this paper, we propose a unified approach that considers balancing both precision and recall by making use of a weighted loss function and an optimization algorithm to determine the best classifier based on F-measure. The performance of this approach on few popular datasets provide evidence on the value of the proposed approach.