DASH: Warm-Starting Neural Network Training in Stationary Settings without Loss of Plasticity
Warm-starting neural network training by initializing networks with previously learned weights is appealing, as practical neural networks are often deployed under a continuous influx of new data. However, it often leads to loss of plasticity, where the network loses its ability to learn new information, resulting in worse generalization than training from scratch. This occurs even under stationary data distributions, and its underlying mechanism is poorly understood. We develop a framework emulating real-world neural network training and identify noise memorization as the primary cause of plasticity loss when warm-starting on stationary data. Motivated by this, we propose Direction-Aware SHrinking (DASH), a method aiming to mitigate plasticity loss by selectively forgetting memorized noise while preserving learned features. We validate our approach on vision tasks, demonstrating improvements in test accuracy and training efficiency.
Paper
Similar papers
Peer review
Summary
This paper examines a setting in which a model is learned with an expanding dataset, such that new data is introduced after the model hits a training accuracy threshold. In particular, the new data comes from the same distribution as the existing data (or sometimes has the exact same statistics), making this a stationary learning problem (unlike the non-stationary problems commonly explored in plasticity loss settings). After demonstrating that plasticity loss occurs in this setting, the paper proposes a new method, DASH, to tackle the problem. DASH shrinks weights based on the cosine similarity between the weight vector and the negative gradient of the training loss. There are a series of experiments which generally show DASH outperforming other plasticity loss prevention approaches, with more in the appendix.
Strengths
- Based on my knowledge and research, I think this is a novel approach to preventing plasticity loss. It is quite original in its idea, which the authors have tried to ground in theory. - This is a relatively underexplored area (plasticity loss under stationary settings), and thus is useful for understanding plasticity loss further. - DASH seems to offer genuine performance improvements over most methods, though it is worth noting that this is not always the case (i.e., in the appendix more results are included which are either significantly more marginal, or sometimes favouring other methods). In particular, the focus that test accuracy is not only improved, but we carry over some of the benefits of warm starting (i.e., quicker convergence) is notable. - The figures/graphs in the paper were generally good and useful for understanding what was being discussed.
Weaknesses
Before noting weaknesses of the paper, which I generally feel come down to clarity issues. I am open to increasing my score in light of discussion based on the author's response to my highlighted weaknesses and questions. - I found a lot of the theory quite confusing, and couldn't follow section 3 ('Comparison Between Warm-Start and Cold-Start'). Similarly, I found some explanations about the method, and the training process quite confusing. Perhaps demonstrating with examples could be useful, but it is currently quite hard to parse exactly what the problem setting is. - This work has not been studied in a particularly standard learning process. Instead, the focus is on a setting where features are learned based on their frequency of appearance in data, rather than their signal strength. This is quite an artificial setting, and it is not exactly clear to me that DASH scales to more conventional learning frameworks. - Appendix A.3 feels slightly burrowed away, but considers the SOTA setting for CIFAR-10 and CIFAR-100, in which DASH underperforms random initialisation. I think this is quite a crucial result, and arguably outweights the results from Table 1 in which a more naive approach is taken. While it is good that the authors include these results in the appendix, it would be good to try to include this in a more upfront way (perhaps including discussion about how DASH is helpful in a naive setting and helps reduce the barrier to entry with less studied datasets?)
Questions
- Considering a different learning setting, where data is continuously added rather than in predictable batches, would DASH still be able to perform (given at the moment, it is applied whenever new data is provided)? Similarly, how is its performance when we are not adding data from a stationary distribution, given this is possibly more realistic? - How do the authors think DASH could work in the more generic learning problem (i.e., where we are not learning features based on their frequency)? - What is actually meant by the frequency of features - can you offer an example please? I find it hard to determine exactly what is meant by the frequency of these discrete features? - In table 8, the authors write CIF1R rather than CIFAR. - What is meant in line 325: 'This allows weights to easily change its direction.'?
Rating
5
Confidence
3
Soundness
2
Presentation
2
Contribution
2
Limitations
- In the appendix, there are examples where DASH does not outperform other methods, or is within the error margin. However, there is clearly a strong signal that DASH on average provides benefit to the training in all but the SOTA setting. The authors are upfront about this. - I believe the authors are generally upfront about the limitations of their work, though I feel perhaps highlighting that they are working in a slightly artificial setting, and demonstrating how DASH could be implemented to a typical machine learning pipeline, would be useful.
Thank you for your feedback and for reconsidering the score. - While we currently don't have the time to fine-tune hyperparameters for DASH in the SOTA setting, we intend to address this in our next revision. In the upcoming update, we will perform more comprehensive hyperparameter tuning, tailored specifically to each method, and incorporate these improved results into our findings. - Thank you for your additional comments on our feature learning framework. We now understand the concerns you raised in W2 and Q2. As you pointed out, the learning of features can also be influenced by the strength of the features, especially when distinct features vary in strength. Our theoretical results and framework can be directly extended to this scenario by slightly altering our approach to consider $(\text{frequency}) \times \text{(strength)}$ as learning criteria instead of just frequency. Additionally, we believe our analysis can be extended to scenarios where feature strength varies across data by treating the set of features as a multiset, where multiple instances of the same element are allowed. Since the analyses in these cases are nearly identical to the one we have considered and follows the same underlying principles, we have assumed all features have identical strength for notational simplicity, without loss of generality. We hope these discussions adequately adress your concerns. We plan to include these discussions as a remark in the next revision to provide a clearer explanation to readers of our learning framework. We welcome any further questions or comments you may have.
Summary
The paper examines warm-start and the loss of plasticity, identifying that noise memorization from warm-start impairs generalization in stationary data distributions. It proposes a method called DASH, which selectively forgets previously memorized noise while preserving learned features.
Strengths
- **Feature Learning Framework**: The paper introduces a feature learning framework to investigate the warm-start issue. It thoroughly explores the properties of warm-start and cold-start training, the role of noise memorization in the loss of plasticity, and the selective retention (forgetting) of features (noise). These core findings are well-supported through theoretical grounding and experimentation. - **Performance:** In vanilla settings, DASH significantly outperforms other methods on several small datasets in terms of accuracy and training steps. And, in the SoTA setting, DASH rivals other methods.
Weaknesses
- **Compute Overhead**: Gradient-based approaches typically incur significant compute overhead. While DASH is claimed to achieve faster convergence, it remains unclear if its shrinking process slows down the overall training. Quantitative comparisons in terms of FLOPs and training time would be insightful. Furthermore, DASH has not been compared with baselines regarding compute and memory overhead. The training convergence based on the gradient norm does not consider the compute and memory overhead associated with DASH’s gradient calculations. - **Assumption: Gradient norm as a proxy for the number of training steps**: The assumption that the number of training steps can be reliably approximated by the initial gradient norm is insufficiently supported by the data presented in Figure 10 of the paper. The correlation between training steps and the initial gradient norm does not appear to be linear, particularly for the Warm start. - **In-depth Analysis**: The experiments do not demonstrate a direct link between learned/memorized data and the cosine similarity of gradients and weights. Conducting ablation studies might provide further insights into this relationship. Additionally, the proposed method appears to be a variant of the shrink and perturb approach. - **Evaluation and Scalability**: Evaluation is performed exclusively on toy datasets, leaving DASH’s scalability to larger and more challenging datasets, such as ImageNet-1K, unknown. - **Generality**: In SOTA settings, as shown in Table 7, DASH does not consistently outperform the compared methods, particularly warm initialization. This raises questions about the wider applicability and robustness of DASH. Its efficacy seems to diminish when using augmentations and weight decay.
Questions
- Does the apparent non-linearity in the correlation between initial gradient norms and training steps in Figure 10 change how Figure 1(b) should be interpreted? - In Figure 2, the test accuracy drops with longer pre-training when the model is pre-trained on 50% of the training data and then fine-tuned on the full dataset. Is this phenomenon universal or does it only occur with the CIFAR dataset? Could the authors provide similar results for high-dimensional datasets, such as ImageNet-1K (224x224)? If the network learns the same set of semantic categories during the pre-training and fine-tuning stages, the fine-tuning stage only differs in terms of the number of samples. Is this phenomenon related to the size of the training data? How much do pre-training and fine-tuning datasets differ in terms of distribution shifts? Is the pre-training data class balanced? How does DASH perform in a similar setting? - Does DASH need access to the entire dataset (all chunks), including current and previous datasets/chunks, for computing gradients and shrinking weights during every training step? - In *Line 116*, the set `S_y` is undefined, and may be corrected to `S_c`. - In *Line 134*, `Tau < K` may be corrected to `Tau <= K`, where the number of learned features can also equal the set of learnable class-dependent features, K.
Rating
6
Confidence
4
Soundness
3
Presentation
3
Contribution
3
Limitations
The paper provides little to no detail on the limitations of this work. While it mentions the assumptions of a discrete learning process and the lack of consideration for a hypothesis class, it does not address whether DASH can be extended to larger problem settings. If such limitations exist, they should be explicitly stated by the authors.
Thanks for your response. We are glad to hear that some of your concerns have been addressed. We want to respond to your additional comments regarding our supplementary experiment in the online learning setting. ### **1. Clarification on “Catastrophic Forgetting”** Before directly answering the questions one at a time, we want to clarify what we meant when discussing the reduction of “catastrophic forgetting” in an IID setting. In our experiment setup, we considered an online learning scenario where all data chunks are drawn from an identical distribution. While catastrophic forgetting is typically less severe in this IID setting as you pointed out, the inability to access previously introduced data in later experiments can still lead to significant forgetting issues. This is particularly problematic with shrinking-based methods, as they intentionally force the model to forget previously learned information through the shrinking process [3]. We considered the inability to retain learned information as a form of catastrophic forgetting. We apologize for any confusion this may have caused. To clarify, when we referred to "catastrophic forgetting" in our previous response to Reviewer LBum, we meant to highlight that shrinking-based methods inherently face this “forgetting” issue. Our point was that applying DASH and S&P for every chunk doesn't actually solve this problem. Instead, we attempt to mitigate the issues associated with shrinking-based methods by applying shrinking with some intervals. ### **2. Degradation of DASH and S&P in Figure 8** Every point at which degradation of test accuracy occurs in DASH and S&P coincides with the shrinking point of each method. The reason for different shrinking cycles is that we report the best learning curve (in terms of the last test accuracy) out of all $3\times3=9$ hyperparameter configurations (the interval between shrinkages $\in \\{10, 15, 20\\}$ & the shrinkage factor $\lambda \in \\{0.05, 0.1, 0.3\\}$) for each method. The performance drops immediately after applying parameter modification (i.e., resetting or shrinking) and subsequent performance recovery is common in the literature of resetting techniques [1,2]. Additionally, in our online learning setting, we only used a newly introduced chunk of data for training at each experiment, rather than training on all previously seen data. This resulted in slower accuracy recovery due to the limited amount of training data. Moreover, when applying shrinking (DASH or S&P) too frequently, the performance degradation caused by shrinking outweighs the learning effect from the limited number of new data points, leading to a decrease in overall test accuracy.
Summary
Warm-starting neural networks may lead to poor generalization performance or loss of plasticity, likely due to overfitting. This paper presents a framework that hypothesizes that noise memorization is the primary cause of loss of generalization in warm-starting settings. The authors then present an algorithm motivated by their framework that improves the generalization and plasticity of the neural networks. The algorithm works on a per-neuron level and measures the cosine similarity between the negative gradient of the input weight to each neuron and its weights. This cosine similarity is used as a utility measure to allow for more shrinking for the input weights with low cosine similarity. The authors showed the effectiveness of their method on a wide range of problems and compared it against other baselines demonstrating superior performance.
Strengths
The paper addresses an important problem: maintaining plasticity and generalizing neural networks. The proposed algorithm is novel and effective, and the experimental results seem comprehensive.
Weaknesses
- The main problem of this work is that the link between the framework and the algorithm is not clear. For example, it’s unclear why different parts of the algorithm are designed this way, especially the cosine similarity part. Why is the momentum direction assumed to be the direction of features? - The evaluation may be unfair and statistically insignificant. In multiple experiments (e.g., figure 1), the results are generated based on three seeds. This is a very low number to have statistically significant results. Additionally, the reported methods use the same hyperparameter across multiple methods, which may be unfair to the baselines since they are not well-tuned compared to DASH. The results should be based on the best hyperparameter of each method. This might explain why the authors found approaches such as L2 Init, resetting, layer normalization, SAM, or reviving dead neurons to be less effective. In contrast, the only effective one was S&P, which uses a hyperparameter (shrinking) similar to the DASH method.
Questions
- In Figure 4, the authors presented an arrow with a feature direction. What does that mean exactly in a rigorous way? - What is the definition of a feature? The word is used in the paper many times without a proper definition, leading to further confusion. In neural networks, the learner always improves its set of features, so they are always changing.
Rating
5
Confidence
2
Soundness
3
Presentation
3
Contribution
3
Limitations
No. I think the main limitation of the method is that the cosine-similarity metric seems to be heuristic-based and not motivated by the introduced framework. It is unclear if such a metric is ideal or if it reflects what is suggested by the theoretical framework.
Summary
This paper investigates the reasons why warm-starting a neural network by pre-training it on a subset of the full training set leads to suboptimal performance when compared to training it on the full dataset from scratch. In particular, it proposes an abstract combinatorial model of feature learning that according to the authors captures the essence of this phenomenon. In this model, warm-starting leads to overfitting on feature noise that hinders learning new features when fine-tuning on the full distribution when compared to cold-starting. The authors then show that there exist an idealized learning algorithm in this abstract setting that can perform as well as cold-starting by forgeting the memorized noise prior to fine-tuning. Based on this theoretical intuition, they then propose Direction-Aware SHrinking (DASH) a technique that shrinks the learning step per-neuron based on its alignment with previous warm-starting epochs. DASH seems to outperform other techniques in a synthetic warm-starting benchmark.
Strengths
1. **Intuitive and insightful theoretical model**: Personally, I find the discrete abstract model of learning of this paper to be very insightful. Theorems 3.4 and Theorems 4.1 are good examples of theoretical propositions that can help cement our empirical observations in deep learning. This model may not provide a rigorous path towards mathematically "proving" the observed warm-starting phenomenon, but it is useful to reason about it. 2. **Convincing explanations of warm-starting**: The provided explanations for the observed behavior of deep models under warm starting and the evidence to support them are solid and convincing. 3. **Good performance of DASH**: DASH seems to perform strongly against prior work on the studied warm-starting benchmarks.
Weaknesses
1. **Unclear details of DASH**: The description of DASH in section 4.5 is not clear and the algorithm listing does not fully explain the algorithm (see questions). This makes it hard to assess the complexity of implementing DASH and its practical relevance. 2. **Limited evaluation of DASH in online learning**: The authors argue that one of the main motivations of studying the decrease of performance under warm-starting is the fact that many networks are trained in an online setting with new data being collected sequentially. However, the only evaluation of DASH is a synthetic warm-starting benchmark that only tests a narrow and unrealistic setting. I am not against this particular benchmark, but I believe the paper would have benefited from further investigations of DASH in a more realistic online setting (including comparing it against strong baselines in that space). 3. **Reasons for improved performance of DASH over cold-starting are not explained**: As the authors highlight, DASH seems to outperform cold-starting in their benchmarks. This phenomenon, however, cannot be explained by their theoretical model and thus it is a weakness of the model itself.
Questions
I would appreciate a deeper description of how DASH works: - Is the running average of gradients computed at every step or every epoch? - Is the shrinking performed after every step, every epoch, every experiment? - Can DASH be combined with any off-the-shelf deep learning optimizer? If so, how? - What is the memory complexity of DASH? If these questions are addressed and the explanations are clear and made part of the final manuscript I would be open to increase my score.
Rating
7
Confidence
3
Soundness
3
Presentation
2
Contribution
3
Limitations
The authors briefly mention mention some of the limitations of their work (such as the unexplained outperformance of DASH over cold-starting) or specificity of their results to the stationary setting.
Dear Reviewer LBum, Thank you for taking the time to review our work and for providing such insightful and constructive feedback. We understand that you may have a busy schedule, but we wanted to follow up to ensure that our responses have sufficiently addressed your concerns. If you have any further questions or comments, we would be glad to hear them.
Thank you for your rebuttal
I thank the authors for answering my questions and providing a thorough response to all reviewers. After reading the other reviews and the authors's responses I have decided to increase my score to 7: Accept as the authors have clarified the main details that were unclear in the implementation of DASH and provided new insightful experiments in a different online learning scenario (I apologize for my confusion in my previous review and acknowledge that their previous setting was also realistic for a data accumulation setup). I would encourage the authors to improve the description of DASH in a potential camera-ready version in any case.
Thank you for your feedback and for reconsidering the score. We are glad to hear that our response addressed your concerns. We would also be happy to hear if you have any additional thoughts or suggestions. Best regards, Authors
I thank the authors for their detailed responses, clarifications, and additional experiments. Some of my concerns are addressed by the authors. I have a few more questions and concerns. Although the concern regarding online learning is raised by reviewer LBum, I also need clarification regarding this. - Regarding Figure 8 in the attached PDF, could you explain why the test accuracy of DASH and S&P sometimes decreases and is less smooth compared to the "warm" method? Additionally, can you confirm that the model is only trained on new data or the new chunk, rather than on all previously seen chunks? - In your online learning experiments (Figure 8), I assume each chunk contains almost all classes from the CIFAR-10 dataset. I’m interested in understanding whether there is any distribution shift between subsequent chunks. In continual learning, there is a setting called class incremental learning (CIL), where chunks or batches of data are mutually exclusive, meaning classes do not overlap across chunks. For example, if CIFAR-10 is divided into 10 chunks, each chunk would contain only one unique class. If DASH were applied in this setting, how effective would shrinking by cosine similarity be? Could you explain this in terms of learned features and noise? - In contrast, there’s another setting in continual learning called IID (independent and identically distributed), which is the opposite of CIL. In the IID setting, chunks or batches contain overlapping classes, leading to minimal distribution shift, unlike in CIL where the shift is significant. Consequently, catastrophic forgetting is significant in CIL and negligible in IID. I assume all your experimental settings in the paper and rebuttal are based on IID, is that correct? - My primary question is: Is DASH only applicable in scenarios where subsequent chunks/batches/experiments contain the same/ overlapping classes (IID setting)? If so, it cannot be claimed that DASH addresses catastrophic forgetting. It would also be helpful to include S&P in Figure 3 of the attached PDF. I suggest incorporating the additional experiments and results into the paper and clarifying the applicability of DASH in response to my questions.
### **3-1. Explanation of DASH on Online CIL Setting** We acknowledge that Class Incremental Learning (CIL) is an important research topic in continual learning where data distribution is non-stationary. Following your suggestion, we conducted experiments to explore DASH's effectiveness in a CIL setting, which contrasts with our original setup. We divided CIFAR-10 into 10 chunks, each containing data from a single class. In this setting, at the start of each experiment, a chunk with a specific label is introduced, and the model is trained using only this new chunk without access to previous data. We trained CIFAR-10 on ResNet-18 with five random seeds. Please let us know if you have a different setup in mind. We first conducted experiments with warm-starting as a baseline in this setting. The results showed that the model overfits the data introduced in each experiment, predicting only the corresponding label, resulting in a test accuracy of around 10% for all experiments. In this online learning scenario, it seems unlikely that any method manipulating the initialization at the beginning of every experiment, including warm-starting, DASH, or S&P, would be effective. In fact, both DASH and S&P performed no better than random guessing, also achieving only 10% test accuracy, simillar to warm-starting. Instead, we might need to modify the model architecture or employ other training techniques e.g. replay buffer or regularization. Despite this, it's worth considering why DASH performs as bad as warm-starting. The primary reason is the limited overlap between class-dependent features across different data chunks. If we apply DASH at the beginning of each experiment, even neurons that learned features from previous chunks would have low cosine similarity with the negative gradients of the new chunk. This leads to significant shrinking and forgetting. It's important to note that our algorithm was originally designed for stationary settings. As we mentioned in our response to rFKm, non-stationary cases require different considerations. We view this as an important direction for future work. ### **3-2. CIL with “Data Accumulation” Setup** The setting described above is not suitable for evaluating DASH's applicability, as even warm-starting performs no better than random guessing. Given that even a warm-started neural network struggles in a standard CIL setting, we conducted additional experiments in a modified CIL environment. Similar to the previous setup, we divided CIFAR-10 into 10 chunks, each containing data from a single class. This time, instead of discarding data from previous experiments, we combined newly introduced data with the existing training data (i.e., the same as the “data accumulation” setup considered in our paper). This approach aligns more closely with our main focus while still maintaining the core concept of non-stationarity you mentioned. During testing, we evaluated the model only on the labels it had encountered during training. The model trained with warm-starting achieved a final accuracy of $59.55\% (\pm 0.53\%)$ in this setting, with an average test accuracy of $72.39\% (\pm 0.53\%)$ across 10 experiments. In contrast, when applying DASH, we achieved a final accuracy of $83.74\% (\pm 0.45\%)$ with an average test accuracy of $87.86\% (\pm 0.18\%)$. These results show that DASH can be effective in certain non-stationary settings, though our modified setup doesn't fully represent a truly non-stationary environment. While we can't entirely explain DASH's strong performance in this context with our abstract theoretical framework (Section 2), we attribute it to two key factors. The first factor is our experimental approach. By incorporating data from previous experiments, we allow this information to continue contributing to the learning process. This is why warm-starting tends to yield better test accuracy in this scenario. Another key factor is DASH’s algorithm design. Even when DASH shrinks all weights to their lower bound $\lambda$ because of low cosine similarity, it would likely perform similarly to S&P. ### **4. Regarding IID Setting** You're correct in noting that we focused on the IID setting. We aim to address why warm-starting underperforms in this IID setting (i.e., the loss of plasticity) and to propose a solution to this issue. Thus, we would like to emphasize that the significance of our research in the IID setting remains important and should not be overlooked. ### **5. Regarding S&P in Figure 3 of the attached PDF** Due to the computation-intensive nature of the experiments on ImageNet-1K, we couldn’t make a direct comparison with S&P at this time. We appreciate your suggestion and plan to include this comparison in the revised version of our paper. --- We hope that our response clarifies your concerns. We would appreciate it if you could reconsider your assessment.
**References** [1] Nikishin et al., The Primacy Bias in Deep Reinforcement Learning, ICML 2022 [2] Zhou et al., Fortuitous forgetting in connectionist networks, ICLR 2022 [3] Ash and Adams, On warm-starting neural network training, NeurIPS 2020
I appreciate the authors for providing additional clarifications regarding the IID/CIL settings and the forgetting phenomenon, which overlap in continual learning and the shrinking process. The authors have adequately addressed my major concerns, presenting solid contributions toward resolving warm-starting issues and the loss of plasticity. Therefore, I have raised my rating and believe this paper could be accepted for publication. I suggest the authors clarify these points in the main paper and release the code to ensure reproducibility and transparency.
Dear authors, Firstly, let me apologise for my delay in responding to your rebuttals. I will attempt to remain more active in the coming days to enable active discussion, if necessary. I still hold a few concerns regarding the performance of DASH: - It is quite difficult to make claims about the significance of DASH given the performance in the SOTA setting. Despite the author's beliefs that this may improve with hyperparameter tuning, I would need to see this run experimentally to believe this is the case (I respect that you did not have the time to run this experiment over the rebuttal period due to the strict time constraints). - I am still not certain I believe in even the intuition of the feature learning framework; as I say, this ignores the fact that different signals provide varying levels of signal for learning, and thus the frequency of features is not the only factor that determines learning. That said, additional results provided in the global response have strengthened your paper in my opinion. I also agree that it would be interesting to see how DASH could be applied to nonstationary settings like RL, and would suggest adding this as a proposal for obvious future work. I have raised my score to recommend borderline acceptance.
Thank you for your response
I thank the authors for their detailed response. After reading the response and other reviews, I think my initial assessment has been confirmed. I think major restructuring is needed to build a logical flow between the framework and the algorithm. While the authors explained the intuition, it still lacks some rigor, and judging a completely rewritten introduction and method sections requires another review process. Therefore, I maintain my original score.
Thanks for your response. We acknowledge that the logical flow between the framework and the algorithm is weak in our current draft, which could lead to misunderstandings or make it difficult for readers to follow. To address this, we plan to reorganize the paper as follows: - Add the main intuition behind our theoretical framework which included in our global response at the beginning of Section 2 to make it easier for readers to understand and follow our framework. - Move 'Section 4.1 Motivation: An Idealized Method' to the end of Section 3, which mainly addresses the theoretical framework. This will create a clearer separation between the results on the theoretical framework and the real-world scenarios. We believe this change will help prevent any confusion by clearly distinguishing between the discussion of these two distinct settings. - At the beginning of Section 4, add an additional subsection to provide motivation for DASH by presenting the connection between DASH and the idealized method considered in the theoretical framework. This will help clarify how the theoretical findings relate to our practical proposal, DASH. We believe this reorganization will improve the paper. If you have further concerns or comments, we would be happy to hear them.
Decision
Accept (poster)