Automatic pain assessment is essential in personalized healthcare, mental health treatment and lifestyle-coaching, due to the subjectivity of self-reporting. This study focuses on facial expressions as nonverbal indicators of pain and investigates the use of computer algorithms to estimate pain intensity. Since almost all available pain datasets exhibit significant class asymmetry, the first objective of this work is to focus on the impact of class imbalance and relevant alleviation strategies on the achieved pain classification. To this end, we conducted a comprehensive survey of existing literature, reviewing various classification models, features, class imbalance handling methods, and dataset-splitting techniques. The second objective is to develop a model that classifies pain intensity into three levels by aggregating the existing pain scales. We assessed a variety of machine learning models, in conjunction with hand-crafted geometric features based on fiducial points, appearance features (HOG) and image features learnt by deep learning architectures. In addition, we explored and proposed methods that address class imbalance on data level using various under-sampling strategies and algorithmic level utilizing cost-sensitive learning. In order to enhance model generalization, we investigated various stratified data-splitting strategies on sequence and participant levels, for training both classic and deep learning models, so that class representation remains constant during training, validation and testing. Comparative results are presented using the UNBC McMaster Shoulder Pain dataset, from which conclusions are drawn that may prove useful for future research in the area of automatic pain classification.