From Eliza in 1960s to LSTM in 1997 to Google Brain in 2011 to transformer models in 2107 to GPT3 in 2020 and subsequent ChatGPT and BERT large language models (LLMs), the field of artificial intelligence (AI) not only took giant steps toward an exciting future but are also leaving their mark on the mankind in terms of the massive power these LLMs wield. These models are capable of understanding as well as generating human-like text, which is quite helpful in various applications like natural language processing (NLP) and other interactive applications. Since the advent of the transformer models, an umpteen number of LLMs have surfaced both in the market and in the research community. This paper presents a brief review of some of these significant LLMs focusing on their transformer-based architecture and in-context and reinforcement learning capabilities. It presents a crisp comparison of various LLMs such as GPT-4, Bloom, LLaMA, Gemini, BERT, BART, Grammarly, etc., and covers their contributions to the AI landscape along with the open research challenges as well as issues associated with them segregated into various categories.