Motion prediction is essential for autonomous vehicles to navigate complex environments and anticipate the behavior of other traffic participants. As new driving scenarios emerge, models must be continually updated without retraining from scratch. We propose DECODE, a continual learning framework that starts from a pre-trained generalized model and incrementally expands specialized models for distinct domains. Unlike existing approaches that pursue a single unified model, DECODE explicitly balances specialization and generalization through dynamic model selection. It employs a hypernetwork for parameter generation, which reduces storage costs, and utilizes a normalizing flow for real-time domain inference via likelihood estimation. Outputs from specialized and generalized models are fused using Bayesian uncertainty estimation. This integration ensures optimal performance in familiar conditions while maintaining robustness in novel scenarios. Extensive experiments show DECODE achieves a low forgetting rate of 0.044 and an average minADE of 0.584 m, outperforming prior methods and generalizing well across diverse driving domains. Furthermore, we demonstrate that DECODE can be extended beyond motion prediction to general continual learning tasks such as image classification, showcasing its broad applicability.