Hate speech refers to language expressions that attack individuals or groups based on specific characteristics associated with their identities, causing lasting damage. Social networks have become a pertinent environment for hate speech proliferation since they allow anonymity and maintain a safe distance from aggressors and assaulted victims. With the amount of data published every minute, automatic identification of hate speech using machine learning gathered much attention from academic and industrial researchers. However, as with many natural language processing tasks, the efforts mainly focused on English, and languages like Portuguese remain less explored. Therefore, this paper aims to experiment with different techniques to deal with the challenges associated with low-resource languages in automatic hate speech detection. It evaluates whether knowledge transferred from offensive speech detection as a source task can be effective for hate detection and if the unbalanced data poses an obstacle for a Portuguese pre-trained BERT model, BERTimbau. Experimental results show that transferring learning between tasks does not improve performance and that using balanced data leads to better F1 scores and Cohen’s Kappa.