The field of Autonomous Driving Systems (ADS) is the recent area which drew attention of researchers and developers due to advancement in the computer vision technology with the help of emerging deep learning techniques along with latest designed and developed sensor modalities. Starting from the initial days of automated driving competitions on large scale such as Urban and DARPA Grand challenges to the current time, multiple techniques are proposed and there is establishment of common system architectures. Additionally, ample number of activities in ADS are categorised into sub-categories and there is clear visibility of Deep Learning (DL) techniques having its dominance in most of the sub-categories. Robust ADS is still to be implemented in the urban environment. With the advancement in Convolutional Neural Network (CNN) being a sub category of DL, it has become the most preferred choice to implement the visual activities in numerous modules of ADS which aims to reduce the intervention of human in the driving. In this paper, review of recent published work on ADS having focus on image classification based on CNN along with detection of object is presented.