Malware attacks can result in financial losses for individuals, businesses, and organizations. Rapid growth in cyberattacks in recent years has underscored the critical importance of robust malware detection systems to safeguard digital assets, preserve business continuity, and protect against the potential devastating financial and reputational repercussions of successful malware incursions. Anti-malware software developers and researchers are dedicating their time to detect the malware, but due to new obfuscation techniques, it is becoming hard to detect. A model is proposed based on deep neural network (DNN) for binary and multiclass classification between benign and various malware categories, specifically Trojan horse, spyware, and ransomware in this paper. We utilize the CIC-MalMem-2022 dataset, which contains labeled samples categorized into various types of malware and benign files, to analyze and classify memory-based malware threats. Using principal component analysis (PCA), a number of features are reduced. The result achieved by the model for binary is 99.97% and for multiclass classification F1-score is 0.8102. This demonstrates the model’s efficacy, achieved across the different classes. The study highlights the potential of DNN in enhancing malware detection systems, contributing to more robust cybersecurity defenses.