The Internet of Things (IoT) has transformed the way we communicate in the digital world. This technology facilitates data gathering and sharing across a wide range of devices like appliances and vehicles, equipped with software, sensors creating opportunities for more efficient automated systems. Besides these undeniable benefits, the integrity, confidentiality, and availability of IoT networks are threatened by new complicated security issues. To protect the connected ecosystem from attackers, designing and executing efficient security measures are needed. The dynamicity of IoT networks and their quickly changing nature demand the need of advanced security techniques. Herein lies the potential of Machine Learning (ML) and Deep Learning (DL) to transform our view of security research in IoT. The help of ML and DL-based vulnerability analysis and attack modeling aids security experts and researchers in locating potential flaws and attacker reachability in IoT systems. This book chapter discusses the importance of ML and DL in vulnerability analysis and attack modeling by providing a thorough understanding of security issues. A new taxonomy is proposed by incorporating all the perspectives on vulnerability analysis. Further, the chapter also discusses the most recent developments in attack modeling techniques and the potential for creating novel attack modeling strategies for IoT environment.