Medical image processing play a vital role in disease diagnosis and medical research. The most common image acquisition methods include X-ray, CT, MRI and Ultrasound. Every method has its own advantages and disadvantages. Deep learning Models are widely used by image researchers to get clear view of the medical image and thus the diseases like brain tumor, lung cancer, breast cancer, kidney disease, Glaucoma etc. can be diagnosed correctly. In this paper we have analyzed many research papers which include the structure of CNN. We have also analyzed background of transfer learning and types of different transfer learning techniques. An exhaustive study of different deep learning models, their working and their applications in medical field has been done. Through this paper, researchers will get true insight of popular deep learning models and fractal residual networks in field of medical image processing. In the end how different deep learning models can be effectively applied in detection of tumors in different body parts been discussed.