Continual Traffic Forecasting via Mixture of Experts

Response to Reviewer p8v4

We appreciate the time and effort you have dedicated to reviewing our paper. Your detailed comments have helped us to enhance our work significantly. W1: Firstly, I apologize for any inconvenience caused while reading the paper. The contents of Section 4 are considered essential components of our model, which made it challenging to simplify the content. To aid in understanding our model we have added various sections and discussions in the appendix for a more detailed explanation of the model's architecture. Please refer to the updated paper for this information. W2, Q1: Thank you for your constructive advice. In response to your comments, we have created a new section titled 'Appendix J: Exploring Clustering in Traffic Forecasting and Revisiting the Contributions of Our Proposed Model.' In this section, we examine how clustering methodologies are used in existing traffic forecasting, their purposes, and how our modeling aligns or differs from these methods. We also agree that the time and space complexity of the clustering operation is an important consideration. Reflecting this, we have added 'Appendix L: Complexity Analysis,' where we analyze the time and space complexity of the entire model, including the clustering operation. Please refer to the updated paper for this information. W3: Thank you for your valuable feedback. One limitation of our research, as you pointed out, is its generalization ability. The scarcity of datasets in the field of continual traffic forecasting, a relatively new and under-researched area, poses challenges in fully assessing the generalizability of our model. The dataset we currently use represents a real-world long-term streaming network, encompassing a lengthy period of seven years with new sensors added annually. Our model architecture is specifically tailored for such long-term streaming networks. If we were to create a new dataset suitable for long-term streaming networks, it would likely be challenging to do so quickly in a short period. Most existing datasets used in traffic forecasting are collected over short durations and thus are not directly applicable for evaluating our model tailored for long-term streaming networks. Despite these challenges, we have conducted extensive visualization and experimentation to demonstrate that our model is well-adapted and effectively designed for continual traffic forecasting, exhibiting significant performance improvements over existing models. We are hopeful that as continual traffic forecasting becomes more recognized and researched, a wider variety of datasets and benchmarks will become available, energizing the field and enabling more comprehensive future studies and comparative analyses. This discussion will be included in the newly created 'Appendix M: Limitation and Future Work' section.

Response to Reviewer TPK5

We appreciate the time and effort you have dedicated to reviewing our paper. Your detailed comments have helped us to enhance our work significantly. W1, W4: I agree with your opinion that, although technical insights regarding each methodology are provided in the Introduction and Proposed Method sections of the paper, further efforts to clarify these points are necessary. In response, we have created a new section titled ‘Appendix J: Exploring Clustering in Traffic Forecasting and Revisiting the Contributions of Our Proposed Model’ to reorganize and clarify our contributions. Please refer to the updated paper for this information. W2: In response to your feedback, we have created a new section in the appendix titled 'Appendix M: Limitation and Future Work' where we analyze the current limitations of our model. Please refer to the updated paper for this information. W3: Thank you for the valuable feedback. In response, we have added a new section titled 'Appendix L: Complexity Analysis' where we conduct an analysis of both time complexity and space complexity. As mentioned in the Introduction, the reason for using continual learning is that retraining the entire network is inefficient. Instead, we use continual learning methods to maintain high performance while training only a few newly added nodes. However, in our methodology, when we train the first task, we use the entire dataset, which can be problematic. We have addressed this discussion in a newly created section 'Appendix M: Limitation and Future Work'. Please refer to the updated paper for this information. W5, C3: We have presented experimental results for the core hyperparameters that we believe are crucial in our modeling. Experiments on the number of Experts can be found in the appendix 'Appendix G: Optimizing Expert Selection.' Experiments on the replay ratio for 'Reconstruction-Based Replay' are available in 'Appendix H: Replay Method Analysis: Reconstruction-Based Replay vs Random Sampling.' Experiments on the sampling ratio for 'Forgetting-Resilient Sampling' are detailed in 'Appendix I: Evaluating the Impact of Sampling Ratio on Model Performance.' However, as you suggest, experiments on the sensitivity of various other hyperparameters are also important. We plan to add a section related to parameter sensitivity and will update it for inclusion in the final version of the paper. W6: We have described the dataset in a manner that is typical in the field of traffic forecasting. This can be verified in 'Appendix A: Dataset.' Additionally, details about the experimental settings, such as dataset preprocessing, can be found in 'Appendix C: Evaluation Protocol.' Information on the model's hyperparameter settings is available in 'Appendix D: Implementation Details.' Please refer to these sections for more information. C1: In our research, we focus on a homogeneous graph structure that aligns with the nature of our 1-dimensional traffic flow data. Each node in our graph represents a data point in traffic flow, and our structure does not incorporate heterogeneous elements. The primary aim of our study is to model the changes in traffic flow over time, capturing the patterns and variations within this specific context. As such, the question of addressing heterogeneous graph structures does not directly apply to our work, since our methodology and data are inherently homogeneous. While our current approach is specialized for uniform data types, we recognize the potential for future research to explore and incorporate more complex, heterogeneous graph structures, expanding the scope and applicability of our work. C2: If 'pre-trained clustering' refers to the clustering methodology in the pre-training stage, then such an experiment would indeed be a good way to support our main idea. We appreciate the suggestion and have created a new section titled 'Appendix K: Analysis of the Effects of Clustering in the Pre-training Stage' in response to your comments. In this section, we have conducted experiments and analyzed the results. Please refer to the updated paper for this information.

Response to Reviewer WWUg

We appreciate the time and effort you have dedicated to reviewing our paper. Your detailed comments have helped us to enhance our work significantly. W1: As you have mentioned, our research was conducted on a single dataset. Continual traffic forecasting is a relatively new research field with limited datasets available and a scarcity of baselines. We acknowledge these limitations and have discussed them in the 'Appendix M: Limitation and Future Work' section of our paper. We also address this topic in our response to reviewer p8v4, which we invite you to consult for further details. To mitigate these issues, we have conducted a variety of experiments and visualizations for every component and module proposed in our study. We hope this approach will be considered in your assessment of our work. W2: If by 'existing techniques' you are referring to methods like Clustering, Mixture of Experts (MoE), VAE sampling, Learning without Forgetting (LwF), and Graph Learning that we employed to prevent catastrophic forgetting in continual traffic forecasting, then indeed, these are established methods, and we have duly cited them in our paper. The novelty of our work lies in the exploration and analysis of how these methodologies can be appropriately and effectively adapted to the field of continual traffic forecasting. From this perspective, if you are aware of any other papers employing similar techniques in this context, we would greatly appreciate your sharing them with us. This would further enrich our understanding and the scope of our research. W3: Firstly, we conducted various ablation studies to demonstrate the efficiency and validity of each component in our model. Our analysis goes beyond mere ablation; in 'Appendix E: Analysis of Reconstructor and Predictor Outputs' and 'Appendix F: Visualization of Synthetic Data Generated through Decoders', we provided visualizations of our model. Furthermore, in 'Appendix H: Replay Method Analysis: Reconstruction-Based Replay vs Random Sampling', we showcased the effectiveness of our methods through comparative analysis. In response to feedback from other reviewers, our updated version of the paper includes 'Appendix K: Analysis of the Effects of Clustering in the Pre-training Stage', which presents experiments and analysis on the impact of clustering during the pre-training stage. Given that your review was submitted towards the end of the Discussion period, it may not be feasible to immediately incorporate your feedback. However, we are very open to your suggestions. If you could specify what content you recommend for inclusion in the additional analysis, we will make effort to incorporate it into the final version of our paper.