The proliferation of IoT devices and IoT-Edge-Cloud infrastructures has created the need to deploy artificial intelligence applications on resource-constrained devices to achieve efficient decision-making closer to the edge. Frugal AI is an approach that focuses on optimizing machine learning and AI models to be resource-efficient, cost-effective, and scalable, particularly in environments with limited computational resources or energy constraints. Applying frugal techniques to AI models is challenging because of complex trade-off to consider and the lack of silver-bullet approaches. This contribution covers practical considerations when deploying frugal AI and discusses quality metrics that can be used to ensure its effective application. These are considered in the analysis of results of an experimental evaluation of compression techniques, applied to AI-based operation of, real-life anchored scenarios materializing in smart buildings. Results show how frugal methods balance model quality and size compression. Following discussion overviews opportunities for effective deployment on the edge by reducing model size and accelerating inference.