Domain generation algorithm (DGA) generated domain names are widespread. They are used to evade detection of Command and Control servers and prevent blacklisting of attack infrastructure. Therefore, the effective detection of DGA-generated domain names is an important aspect of cyber defense. In this paper we present a novel approach for detecting malicious DGA domain names using deep learning and Generative Pre-trained Transformer (GPT). We leverage GPT Large Language Model (LLM) to decode the domain name patterns and learn dense vector representations which are utilized by a Convolutional Neural Network (CNN) model for higher accuracy detection of malicious DGA domains. We implemented the GPT-based CNN model and evaluated the system on a publicly available dataset consisting of legitimate and malicious DGA-generated domains, comparing its performance to other widely-used used techniques such as TF-IDF, Bag-of-Words and N-grams. The results of our experiments showed that our proposed method outperformed the other techniques by a significant margin.