ChatGPT follows the GPT (Generative Pre-trained Transformer) architecture which is a Deep Learning model designed specifically for NLP (Natural Language Processing) tasks. ChatGPT is a unique conversational AI model which uses both the concepts of Deep Learning as well as Natural Language processing. The fundamental idea of ChatGPT, an enhanced conversational model following the Transformer architecture, is explored first. The capacity of GPT, a groundbreaking deep learning model, to recognise contextual relationships and produce meaningful text has completely changed the NLP industry. ChatGPT is a cutting-edge conversational AI model built on the Generative Pre-trained Transformer (GPT) architecture, specifically designed for Natural Language Processing (NLP) tasks. This paper explores the internal mechanisms of ChatGPT, including its self-attention framework, pre-training strategies, and fine-tuning methodologies. Unlike traditional surveys, this study integrates a comparative lens by examining the performance and design contrasts between RNN-based systems and Transformer models. It also highlights practical applications, ethical concerns, and performance benchmarks. Furthermore, we delve into key NLP techniques used in ChatGPT such as Byte-Pair Encoding (BPE), contextual embeddings, and beam search decoding. The objective is to provide both a technical and practical understanding of ChatGPT’s conversational capabilities and the foundational architecture that powers them.