Due to the absence of accurate semantic similarity measurement techniques, the evaluation of Automatic Text Summarization (ATS) techniques had been a very challenging task for the researchers. Due to the advent of pre-trained transformer models, the trend of automatic text summaries has shifted from extractive to abstractive summaries. As a result, the most popular evaluation technique, ROUGE, is not able to do the right justice to the evaluation of text summaries. This paper presents the various available evaluation techniques for text summarization along with their pros and cons in detail. The Benchmark Microsoft Research Paraphrase Corpus (MRPC) dataset is used for comparative analysis of various techniques like BERTScore, METEOR, ROUGE-1, ROUGE-2, ROUGE-L, and BLEU using precision, recall, and F-score measures. The results are presented in both tabular and graphical formats along with the detailed analysis.