Reinforcement learning (RL) models have emerged as a promising alternative to traditional, model-based control methods for medical systems. Recently, deep RL techniques have been applied to autonomous glycemic control systems, commonly referred to as Artificial Pancreas (AP) systems, which operate through closed-loop communication between a glucose sensor and an insulin pump. This chapter is an updated summary of a paper originally presented at the ACM Cybersecurity in Healthcare (HealthSec) Workshop in October 2024 [7]. We examine the robustness of RL4BG, a prominent deep RL–based AP controller, against a range of glucose sensor malfunctions. We consider two realistic malfunction classes arising from natural errors or adversarial manipulation: (1) Denial-of-Service that captures worst-case sensor failures, and (2) Subtle manipulations that reflects stealthier, prolonged degradations. Our results demonstrate that this new generation of medical control systems is vulnerable to anomalous sensor inputs in safety-critical settings. These findings underscore the need for adversarially robust training methods when deploying RL-based medical controllers.