Combating fake news and misinformation propagation is a challenging task in\nthe post-truth era. News feed and search algorithms could potentially lead to\nunintentional large-scale propagation of false and fabricated information with\nusers being exposed to algorithmically selected false content. Our research\ninvestigates the effects of an Explainable AI assistant embedded in news review\nplatforms for combating the propagation of fake news. We design a news\nreviewing and sharing interface, create a dataset of news stories, and train\nfour interpretable fake news detection algorithms to study the effects of\nalgorithmic transparency on end-users. We present evaluation results and\nanalysis from multiple controlled crowdsourced studies. For a deeper\nunderstanding of Explainable AI systems, we discuss interactions between user\nengagement, mental model, trust, and performance measures in the process of\nexplaining. The study results indicate that explanations helped participants to\nbuild appropriate mental models of the intelligent assistants in different\nconditions and adjust their trust accordingly for model limitations.\n