The rise of large language models (LLMs) has enabled the generation of highly convincing spam and phishing emails, challenging traditional spam filters. This study evaluates the effectiveness of current spam filtering techniques against LLM-generated emails using the Enron-Spam dataset. We developed a filtering pipeline leveraging Llama3.2 embeddings and various classifiers, including neural networks, SVMs, and random forests. Initial tests revealed poor detection accuracy on AI-generated spam. To address this, we enhanced the dataset with synthetic spam created by advanced LLMs such as Gemma2, Llama3.2, and Phi. This augmentation significantly improved classification performance, demonstrating the value of LLM-generated data in adapting filters to emerging threats. Our findings highlight the growing need to evolve spam detection systems in response to AI-generated content and provide a practical foundation for future advancements in email security.