As the popularity of Transformers in computer vision rises, it is likely that Transformer-based models will become the standard for many types of vision applications. The Transformer-based model’s unmatched scalability and vast quantity of trained data make it tough for literatures with fewer data then less computational skills to apply it. Because of its patch-based representation, Transformer-based models have recently been shown to perform very well, and a study presented ConvMixer to demonstrate this. ConvMixer excels in picture classification, but its wasteful isotropic design makes it unfit for other vision tasks. In this research, we offer HEConvMixer, a patch-based representation network that is both hierarchical and data-efficient. In contrast to the original Transformer-based models, our network features two down sample layers and some simple convolutional blocks in place of the Transformer blocks. Our network was trained from scratch on modest datasets using a single GPU. Our HEConvMixer achieves 98.12% accuracy, which is higher than existing Transformer-based models and ConvNets.