The identification of indigenous fish species is vital in aquatic research, fishery management, and environmental monitoring. Fish classification has important practical significance for both the aquaculture industry and ordinary people. Nevertheless, current deep learning techniques for automatic fish classification lack robust feature extraction capabilities affecting the model performance. This is due to complex textual features, similarity of shapes among fish species, skewness in the fish dataset towards majority classes and limited number of available instances while data collection. To tackle this issue effectively, a model capable of operating in resource-constrained environments while maintaining high accuracy is essential. To this end, this paper makes the following contributions. First, a new dataset is proposed with 92 classes. The proposed Fish92 dataset consists of \(\backsim 3500\) edible fish instances targeted over Northeast part of India. Second, an in-depth analysis and evaluation of Fish92 dataset characteristics. Six pre-trained state-of-the-art CNN models were used for the dataset evaluation. These models were heavily biased due to class imbalance in the dataset. Hence, four existing debiasing techniques were employed on Fish92 dataset to have a clear insight into the features.