A Survey of Causal Discriminative Neural Networks

Over the past years, there has been an increasing interest in using the ideas of causality to improve discriminative neural network models. Existing survey treatments of this topic have typically been focused on mathematical formalisms. In this paper, we take a more practical approach and present a survey of the state-of-the-art in these causal neural networks, focusing on the architectural and implementational aspects of each model. By analysing the implementations and applications of the different models, we propose a model taxonomy with two orthogonal axes of classification. Firstly, we analyse how each model architecturally represents causal and-where applicablespurious input information. Secondly, we make a distinction based on how models represent and correct for confounders. Based on this taxonomy, we highlight practical similarities and differences between the analysed models and draw connections to the broader field of neural network models. We further analyse how narrow or general each model is in its applicability, and suggest potential future applications. To complement these applications, we present a summary of relevant benchmarking datasets and representation learning metrics. Based on our analysis, we end by highlighting open problems and avenues of future work. We believe that this survey will serve as a succinct and practical guide, both for researchers experienced in causal neural networks and for newcomers from traditional neural network research.