Artificial Intelligence (AI) and Machine Learning (ML) have become pervasive technologies, raising complex ethical challenges. While prior discussions often addressed ethical themes in general, the rapid deployment of Large Language Models (LLMs) presents new, concrete dilemmas. This work critically examines key ethical dimensions, including data privacy, algorithmic bias, transparency, accountability, and sustainability, with a particular focus on their implications for public administration. The regulatory context of the European AI Act provides a reference framework, but significant ethical and governance gaps remain. We propose a structured analysis of the unique risks posed by LLMs in administrative decision-making, and outline actionable recommendations for responsible deployment.