Potatoes are widely grown in India and many parts of the world, making them a key agricultural crop. It is an economical vegetable and a good source of nutrients and energy. Farmers in potato farming face multiple diseases, such as Early Blight (EB) and Late Blight (LB), affecting their crop income and quality. In the present scenario, farmers and agronomists use traditional or manual methods like visual inspection to reduce these disease challenges, which delays results and increases the risk of inaccuracies for large land areas. This study highlights the use of deep learning methods to identify potato leaf disease in the publicly available Potato Leaf Disease (PLD) dataset. In this research, we proposed a custom convolutional neural network (CNN) model that outperformed all state-of-the-art methods and achieved an accuracy of 98.70% on the PLD dataset. The work is beneficial for managing large-scale potato production, resulting in healthier crops, a stable food supply, and improved economic prospects for farmers.