Classification of fruits into different classes based on their species, cultivars, shape and other aspects has been performed for a very long time. However, these processes had been largely manual in nature and were inhabited with various concerns including laborious processes, time consuming, contamination due to human handling and others. There is a need for the development of an automated system for classification of fruits. Combined strength of image processing and machine learning tools are a boon towards development of such systems. In this paper, a dataset of images of these fruits are developed containing a total of 9600 images for three different types of fruits including apples, bananas, and oranges. Different machine learning algorithms including Support Vector Machine (SVM), Decision Tree (DT), K-Nearest Neighbour (KNN) Random Forest (RF) and Convolutional Neural Network (CNN) are employed for classification of these fruits into their respective classes. After simulation of these models, DT and RF are observed to provide highest classification accuracy of 98.47 and 98.63% respectively. Two convolutional layer based CNNs achieved the accuracy of 97.33%. Therefore, it is recommended to use DT or RF based classifiers for segregating different fruits into their respective classes.