This study investigates the impact of class imbalance and its potential interplay with other factors on machine learning models for transportation mode classification, utilising two real-world GPS trajectory datasets. A Random Forest model serves as the baseline, demonstrating strong performance on the relatively balanced dataset but experiencing significant degradation on the imbalanced one. To mitigate this effect, we explore various state-of-the-art class imbalance learning techniques, finding only marginal improvements. Resampling the fairly balanced dataset to replicate the imbalanced distribution suggests that factors beyond class imbalance are at play. We hypothesise and provide preliminary evidence for class overlap as a potential contributing factor, underscoring the need for further investigation into the broader range of classification difficulty factors. Our findings highlight the importance of balanced class distributions and a deeper understanding of factors such as class overlap in developing robust and generalisable models for transportation mode detection.