Presence of interictal activity projects the diagnosis of neurological problems, prominently epilepsy. Deep learning techniques have been recently used for classification of medical images. Identifying interictal activity for diagnosis of epilepsy has been a challenge for researchers. Identifying these correctly supports the diagnosis of neurologists for epilepsy prominently and also some other disorders. In this work, we have compared the performance of different deep learning techniques in identifying interictal activity amidst of artifacts. The data is real time data collected from Max Hospital, Saket. Firstly, data is divided into two second segments and converted into scalograms. Using these scalograms as input the work tries to distinguish between different artifacts and interictal activity. The automation of this process will support the neurologist as artifacts are prominent in an electroencephalography test, which is preferred by most patients due to its non-invasive nature. Among the models, LeNet stands out as the best performer with the highest scores across Accuracy, Precision, Recall, and Specificity both in K-fold cross-validation and testing. Google Net and Efficient Net also perform well in terms of Recall but lag in other metrics. Shuffle Net and ResNet50 show lower overall performance, especially in Specificity and Recall, respectively.