Scholarlink

This chapter reviews the application of Markov Decision Processes (MDPs) for minimising the Age of Information (AoI) in Unmanned Aerial Vehicle (UAV)-assisted wireless sensor networks (WSN) and Internet of Things (IoT) networks. MDPs provide a robust framework for sequential decision-making, especially well-suited for reinforcement learning (RL)-based solutions. The surveyed literature formulates these problems as either fully observable MDPs or, less commonly, partially observable MDPs (POMDPs). Key optimisation goals investigated across these frameworks include UAV trajectory, energy consumption, data scheduling, altitude control, and packet loss. We systematically analyse 20 papers focused on MDPs and three on POMDPs, categorized by their optimisation objectives, revealing the prevalence of fully observable models in this research area.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

AoI-Aware UAV-IoT Modeling Using MDPs

  • 章节

  • 发表日期：

    2025年

查看全文
  • Oluwatosin Ahmed Amodu,
  • Raja Azlina Raja Mahmood,
  • Huda Althumali,
  • Umar Ali Bukar,
  • Nor Fadzilah Abdullah,
  • Chedia Jarray

摘要

This chapter reviews the application of Markov Decision Processes (MDPs) for minimising the Age of Information (AoI) in Unmanned Aerial Vehicle (UAV)-assisted wireless sensor networks (WSN) and Internet of Things (IoT) networks. MDPs provide a robust framework for sequential decision-making, especially well-suited for reinforcement learning (RL)-based solutions. The surveyed literature formulates these problems as either fully observable MDPs or, less commonly, partially observable MDPs (POMDPs). Key optimisation goals investigated across these frameworks include UAV trajectory, energy consumption, data scheduling, altitude control, and packet loss. We systematically analyse 20 papers focused on MDPs and three on POMDPs, categorized by their optimisation objectives, revealing the prevalence of fully observable models in this research area.