Cassava is a strategic food security crop and affected by diseases such as Cassava Mosaic Disease and Cassava Bacterial Blight, which negatively affect agricultural yield. The traditional way of doing the tiresome detection process shows the necessity of an automatic deep learning process. But the factors related to optimization and scalability still pose a challenge to the model. Our work explores the efficient selection of deep learning models for cassava disease identification by comparing performance metrics under different computational scenarios and with different dataset sizes. Multiple deep learning architectures were evaluated using a small dataset and a large data set with 5 diseases classes. It was found that for the smaller dataset DenseNet121 is the most accurate with accuracy of 90.5%. Extended scalability experiment with MobileNetV2, InceptionV3 and VGG19-BN on the larger data set confirmed MobileNetV2 adaptation at better 85.06% training, and 83.90% validation accuracy rate. The research offers strategic recommendations for choosing deep learning models by considering the trade-off between high accuracy and available computational resources for practical deployment.