The Internet of Things (IoT) is growing rapidly. IoT adoption and expansion have contributed to a sharp rise in IoT malware attacks. IoT malware is malicious software that causes harm to IoT devices by performing unauthorized actions that can harm the data, intrude on privacy, and exploit the system. Detection of IoT malware is the process of finding and preventing harmful software from infecting IoT devices. The existing methods for IoT malware detection do not focus on discriminative features. Moreover, the existing methods may not be able to extract the comprehensive set of features while detecting the malware due to its reliance on a single feature extractor. To this end, the proposed DL model uses a comprehensive set of features from different models using Ensemble Learning, which provides a variety of features for the detection of malware. For feature extraction, seven CNN models are used, namely VGG16, VGG19, ResNet50, DenseNet201, MobileNetV2, XceptionNet, and EfficientNetB7. A multi-head attention mechanism is employed that focuses on the features extracted from the individual models to bring out the discriminative information. The proposed method uses a label computation method based on majority voting and confidence decision. The study is done on the benchmark IOT_Malware dataset. The proposed model has four modules: Feature Extractor, Multi-Head Self-Attention, Deep Learning Network, and Label Computation. To evaluate the model’s capability to detect the malware, we have used accuracy, precision, recall, F1-score, and Matthews Correlation Coefficient (MCC) as the performance metrics. The proposed model achieved an accuracy of 98.11%, F1-score of 98.9%, and MCC of 92.27%. Additionally, a comparative study with other CNN methods is performed; the study demonstrates that the proposed method outperforms the compared methods.