A significant percentage of the information within Electronic Health Records (EHR) consists of unorganized language that contains information regarding clinical events. The extraction of this information is necessary to facilitate subsequent analysis and application in both everyday healthcare settings and research endeavors. A key issue in Natural Language Processing (NLP) is clinical Named Entity Recognition (NER), which entails identifying and capturing key ideas (named items) from medical narratives. It identifies major concepts like frequency, dosage, duration, etc. from the medical text. In this work, the i2b2 2018 track2 dataset has been annotated to assign entity labels to individual tokens in a phrase, therefore increasing the quantity of annotated data and enhancing the process of extracting entities. A comparison of extant techniques for the automated extraction of clinical entities from medical records is presented in this article. The experimental findings demonstrate that our model outperforms state-of-the-art by an accuracy improvement of 2.35% with Bidirectional Encoder Representations from Transformers (BERT) based techniques.