Clustering high-dimensional unlabelled data is a challenging task. We propose a probabilistic generative model based on the variational autoencoder (VAE) that learns the underlying statistical distribution of the dataset and performs cluster analysis. We assume a mixture distribution for both the posterior and prior components and derive the evidence lower bound of our mixtures VAE algorithm, which integrates the clustering distribution within each component of the VAE framework. We explicitly use the EM algorithm to find the clustering assignment estimate and model parameters. We also propose a constrained version of the mixtures VAE model to balance the reconstruction and regularization components during optimization. The experimental results of our proposed model demonstrate superior clustering performance compared to baseline algorithms. Moreover, the proposed model generates realistic examples from specified clusters in the latent space.