Sequential Domain Adaptation through Elastic Weight Consolidation for Sentiment Analysis

Elastic Weight Consolidation (EWC) is a technique used in overcoming\ncatastrophic forgetting between successive tasks trained on a neural network.\nWe use this phenomenon of information sharing between tasks for domain\nadaptation. Training data for tasks such as sentiment analysis (SA) may not be\nfairly represented across multiple domains. Domain Adaptation (DA) aims to\nbuild algorithms that leverage information from source domains to facilitate\nperformance on an unseen target domain. We propose a model-independent\nframework - Sequential Domain Adaptation (SDA). SDA draws on EWC for training\non successive source domains to move towards a general domain solution, thereby\nsolving the problem of domain adaptation. We test SDA on convolutional,\nrecurrent, and attention-based architectures. Our experiments show that the\nproposed framework enables simple architectures such as CNNs to outperform\ncomplex state-of-the-art models in domain adaptation of SA. In addition, we\nobserve that the effectiveness of a harder first Anti-Curriculum ordering of\nsource domains leads to maximum performance.\n