In this paper, we propose a novel semi-supervised subspace learning method to learn a low-dimensional subspace for outlier detection on big data. We propose Hybrid Deep Support Vector Data Description (HDSVDD), a linear Deep Neural Network (DNN) trained on inliers only, to learn a mapping to a lower-dimensional representation. The training procedure of HDSVDD jointly trains connected deep linear neural networks, aiming to map all inliers into compact lower-dimensional hyperspheres. Then, the low-dimensional coordinates of the points are calculated based on the distance of the point to each of the hypersphere centers. The one-class training procedure of the method is expected to facilitate the discrimination between inliers and outliers by mapping the inliers closer to the hyperspheres’ centers and the outliers further apart, based on the assumption that the outliers differ from the (inlier) observations used for training. The experimental results show that the subspace learned by the proposed method can be effectively used for outlier detection with various unsupervised and (semi-)supervised outlier detection methods, improving the outlier detection performance in most cases and maintaining it in the others, while making the outlier detection process more efficient.