Medical question-answering is vital in the era where medical knowledge is expanding yet extracting contextually relevant information from vast medical text data poses research challenges. This study presents an advanced transformers-based model tailored to the intricacies of medical text. Addressing the need for context-aware and domain-specific answers, the proposed model leverages a range of embeddings, including BioBERT, segment embeddings, and relative positional embeddings, to decipher the nuances of the medical domain. Experimenting across diverse datasets encompassing single-span and multi-span questions, the balance between accuracy and F1-score is explored, offering metric insights. The empirical results reveal BioBERT’s superior efficacy and the model’s adaptability, which is poised to transform medical information retrieval and advance healthcare research. This study serves as a beacon and illuminates the path toward natural language understanding within the medical text domain. The future holds opportunities for fine-tuning, scaling, and pushing the limits of our capabilities as we unearth the hidden treasures concealed within the vast tapestry of medical knowledge.