Cancer is one of the most severe diseases in human beings. Early-stage and precise diagnosis by an automated system can lead to a reduction in the high mortality rate by improvising and speeding up the diagnostic procedure. Our proposed study demonstrates the performance of EfficientNet architectures, eventually obtaining a model that outperforms the state-of-the-art models in the H&E-based histopathological liver and colon cancer classification domain. For the proposed study, 3920 extracted patches from the publicly available Cancer Genome Atlas Hepatocellular Carcinoma repository formed the primary dataset. The publicly accessible LC25000 constitutes a colon cancer dataset. EfficientNetb0 was found to outperform other models after extensive experimentation with different EfficientNet architectures (EfficientNetb0 - EfficientNetb7) on both datasets. Furthermore, the proposed model is implemented on an embedded board, the Raspberry Pi-B4, by deploying EfficientNetb0, which takes an average of 1.810 s for inference.