Deep Fakes are now a concern and a threat to society as they are mostly misused for creating chaos and are tools of crimes. There are many software that can create the most realistic deep fakes in a matter of seconds using image processing and deep learning techniques. This work deals with and adapts transfer learning techniques for the detection of deep fake videos. It also proposes the ensemble model advantages for better judgment. The FaceForensics++ dataset which consists of FaceSwap, Face2Face, NeuralTextures, DeepFakes, and small portions of them are used for training three different transfer learning models VGG19, EfficientNetB5, and InceptionResNetV2. The ensemble model of these is used to finally communicate the results. The results obtained individual model-wise as well as ensemble of the same has been observed as part of this work. The detection accuracy of InceptionResNetV2 is 96.74% whereas the ensemble model is 65% for the same samples and training parameters. The results of our work indicate that adaptation of transfer learning is a good sign for deep fake video detection and it performs better when trained with large amounts of heterogenous datasets.