In multilingual societies like the Indian subcontinent, use of code-switched\nlanguages is much popular and convenient for the users. In this paper, we study\noffense and abuse detection in the code-switched pair of Hindi and English\n(i.e. Hinglish), the pair that is the most spoken. The task is made difficult\ndue to non-fixed grammar, vocabulary, semantics and spellings of Hinglish\nlanguage. We apply transfer learning and make a LSTM based model for hate\nspeech classification. This model surpasses the performance shown by the\ncurrent best models to establish itself as the state-of-the-art in the\nunexplored domain of Hinglish offensive text classification.We also release our\nmodel and the embeddings trained for research purposes\n