The Medicare and Medicaid programs are needed to help manage the high costs associated with quality healthcare. Unfortunately, there are individuals who commit fraud for nefarious reasons and personal gain, limiting Medicare's ability to effectively provide for the healthcare needs of the elderly and other qualifying people. Applying machine learning models in the fraud detection space often results in a trade-off between performance and interpretability. Models that have better predictive performance tend to be less interpretable, which contrasts with models that are more interpretable but have comparatively worse predictive performance. Unlike other industry sectors that typically focus on predictive performance, the medical industry has a unique demand for machine learning models that can perform in both areas. This paper explores the application of various machine learning models to a Medicare dataset, with a focus on partially and fully interpretable approaches. This study also focuses on analyzing local and global feature explanations to enhance fraud detection accuracy, offering insights essential for refining healthcare fraud prevention strategies.