The rapid expansion of the Internet of Things (IoT) has brought significant security challenges, primarily due to vulnerabilities in the firmware of IoT and network devices, which is predominantly written in low-level programming languages such as C and C++. Traditional vulnerability detection techniques, including static and dynamic analysis, rely on manual rules and face limitations with modern software complexity. Machine learning and deep learning offer promising alternatives by identifying vulnerability patterns from data but require extensive labeled datasets and may struggle to generalize to new code patterns, particularly in the diverse and evolving IoT landscape. This work evaluates open-source Large Language Models (LLMs) for zero-shot vulnerability detection in C and C++ using prompt engineering, benchmarking their performance against traditional static analysis tools and deep learning methods. Leveraging a newly introduced real-world test set, our experimental results confirm the inherent complexity of the vulnerability detection task and highlight the importance of further research aimed at enhancing the effectiveness of existing approaches in realistic contexts.