Unlocking One-for-All Generative AI at Edge
摘要
The rapid advancement of Large Language Models (LLMs) has ushered in a new era of generative AI capabilities, revolutionizing language understanding, reasoning, and multimodal interaction. However, their deployment is largely tethered to cloud infrastructure, limiting accessibility, increasing latency, and raising privacy concerns. With the proliferation of edge devices equipped with specialized accelerators like NPUs, there is a compelling opportunity to shift GenAI to the edge—enabling local, responsive, and private inference. This chapter provides a comprehensive account of architectural, algorithmic, and deployment innovations that unlock generative AI on resource-constrained platforms. We explore model compression techniques, quantization-aware training, runtime optimization, and edge-compatible decoding strategies. We further examine the integration of LoRA-based personalization, federated fine-tuning, and lightweight vision-language models, all within the context of Beyond 5G (B5G) systems. Special attention is given to runtime toolchains, hardware-aware packaging, and communication-efficient inference. This chapter offers a complete blueprint for researchers and practitioners seeking to build responsive, secure, and scalable GenAI on the edge.