The rapid advancement of Large Language Models (LLMs) has ushered in a new era of generative AI capabilities, revolutionizing language understanding, reasoning, and multimodal interaction. However, their deployment is largely tethered to cloud infrastructure, limiting accessibility, increasing latency, and raising privacy concerns. With the proliferation of edge devices equipped with specialized accelerators like NPUs, there is a compelling opportunity to shift GenAI to the edge—enabling local, responsive, and private inference. This chapter provides a comprehensive account of architectural, algorithmic, and deployment innovations that unlock generative AI on resource-constrained platforms. We explore model compression techniques, quantization-aware training, runtime optimization, and edge-compatible decoding strategies. We further examine the integration of LoRA-based personalization, federated fine-tuning, and lightweight vision-language models, all within the context of Beyond 5G (B5G) systems. Special attention is given to runtime toolchains, hardware-aware packaging, and communication-efficient inference. This chapter offers a complete blueprint for researchers and practitioners seeking to build responsive, secure, and scalable GenAI on the edge.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Unlocking One-for-All Generative AI at Edge

  • Srinivas Soumitri Miriyala,
  • Vikram Nelvoy Rajendiran,
  • Sharan Allur

摘要

The rapid advancement of Large Language Models (LLMs) has ushered in a new era of generative AI capabilities, revolutionizing language understanding, reasoning, and multimodal interaction. However, their deployment is largely tethered to cloud infrastructure, limiting accessibility, increasing latency, and raising privacy concerns. With the proliferation of edge devices equipped with specialized accelerators like NPUs, there is a compelling opportunity to shift GenAI to the edge—enabling local, responsive, and private inference. This chapter provides a comprehensive account of architectural, algorithmic, and deployment innovations that unlock generative AI on resource-constrained platforms. We explore model compression techniques, quantization-aware training, runtime optimization, and edge-compatible decoding strategies. We further examine the integration of LoRA-based personalization, federated fine-tuning, and lightweight vision-language models, all within the context of Beyond 5G (B5G) systems. Special attention is given to runtime toolchains, hardware-aware packaging, and communication-efficient inference. This chapter offers a complete blueprint for researchers and practitioners seeking to build responsive, secure, and scalable GenAI on the edge.