The surge of fake accounts on social media platforms and the dissemination of malicious content poses significant cyber threats and escalation of misinformation to online communities. This study focuses on the detection of fake Twitter/X accounts along with identifying malicious URLs distribution from those accounts, utilizing advanced deep learning techniques. The approach employs feature engineering with the Twitter account metadata and user behavioral pattern with a diverse dataset. Deep learning architecture includes a Convolutional Neural Network (CNN) for classifying genuine and fake accounts. Subsequently, an algorithm is developed to find all the URLs from tweets and utilizes a URL-finder service for detecting potential malicious URLs. Various evaluation metrics such as accuracy, precision, recall and F1 score are employed to measure the performance of the deep learning model. Rigorous testing with the proposed model demonstrates robust performance with an accuracy of 95% in distinguishing genuine and fake accounts and identifying potential malicious URLs. This study aims to contribute to the mitigation of deceptive activities on Twitter/X and enhance the security and integrity of social networks.