Generated hateful and toxic content by a portion of users in social media is\na rising phenomenon that motivated researchers to dedicate substantial efforts\nto the challenging direction of hateful content identification. We not only\nneed an efficient automatic hate speech detection model based on advanced\nmachine learning and natural language processing, but also a sufficiently large\namount of annotated data to train a model. The lack of a sufficient amount of\nlabelled hate speech data, along with the existing biases, has been the main\nissue in this domain of research. To address these needs, in this study we\nintroduce a novel transfer learning approach based on an existing pre-trained\nlanguage model called BERT (Bidirectional Encoder Representations from\nTransformers). More specifically, we investigate the ability of BERT at\ncapturing hateful context within social media content by using new fine-tuning\nmethods based on transfer learning. To evaluate our proposed approach, we use\ntwo publicly available datasets that have been annotated for racism, sexism,\nhate, or offensive content on Twitter. The results show that our solution\nobtains considerable performance on these datasets in terms of precision and\nrecall in comparison to existing approaches. Consequently, our model can\ncapture some biases in data annotation and collection process and can\npotentially lead us to a more accurate model.\n