One Network Fits All? Modular versus Monolithic Task Formulations in Neural Networks

Can deep learning solve multiple tasks simultaneously, even when they are\nunrelated and very different? We investigate how the representations of the\nunderlying tasks affect the ability of a single neural network to learn them\njointly. We present theoretical and empirical findings that a single neural\nnetwork is capable of simultaneously learning multiple tasks from a combined\ndata set, for a variety of methods for representing tasks -- for example, when\nthe distinct tasks are encoded by well-separated clusters or decision trees\nover certain task-code attributes. More concretely, we present a novel analysis\nthat shows that families of simple programming-like constructs for the codes\nencoding the tasks are learnable by two-layer neural networks with standard\ntraining. We study more generally how the complexity of learning such combined\ntasks grows with the complexity of the task codes; we find that combining many\ntasks may incur a sample complexity penalty, even though the individual tasks\nare easy to learn. We provide empirical support for the usefulness of the\nlearning bounds by training networks on clusters, decision trees, and SQL-style\naggregation.\n