With surge in online platforms, there has been an upsurge in the user\nengagement on these platforms via comments and reactions. A large portion of\nsuch textual comments are abusive, rude and offensive to the audience. With\nmachine learning systems in-place to check such comments coming onto platform,\nbiases present in the training data gets passed onto the classifier leading to\ndiscrimination against a set of classes, religion and gender. In this work, we\nevaluate different classifiers and feature to estimate the bias in these\nclassifiers along with their performance on downstream task of toxicity\nclassification. Results show that improvement in performance of automatic toxic\ncomment detection models is positively correlated to mitigating biases in these\nmodels. In our work, LSTM with attention mechanism proved to be a better\nmodelling strategy than a CNN model. Further analysis shows that fasttext\nembeddings is marginally preferable than glove embeddings on training models\nfor toxicity comment detection. Deeper analysis reveals the findings that such\nautomatic models are particularly biased to specific identity groups even\nthough the model has a high AUC score. Finally, in effort to mitigate bias in\ntoxicity detection models, a multi-task setup trained with auxiliary task of\ntoxicity sub-types proved to be useful leading to upto 0.26% (6% relative) gain\nin AUC scores.\n