HR-Extreme: A High-Resolution Dataset for Extreme Weather Forecasting

Response to Reviewer tGGo [Part 1]

Thank you for your positive feedback. We truly appreciate your insightful understanding and recognition of the key contributions of our study in terms of significance, comprehensiveness and a strong baseline model for weather prediction on HRRR data! We have revised our paper according to your suggestions and provide our responses to address your concerns below. > **More experiments with SOTA models and HR-Heim. Include prediction for more than a single hour ahead** **A**: Thank you for your suggestions. We have finished the experiment of NWP on HRRR original data, NWP model and HR-Heim on HR-Extreme for lead time from 1 to 4 and added to the end of Section 4.3 in blue. There is also additional statistical analysis in Appendix A.1. The experiments shows HR-Heim consistently outperform others in all lead times. Training and evaluating is highly resource-intensive and time-consuming, however, we want to note that the primary goal of this work is to address the gap in high-resolution datasets for extreme weather evaluation, encompassing comprehensive event types rather than HR-Heim model. We aim to highlight the significance of extreme weather prediction for future research, demonstrate the substantial room for improvement in the performance of current SOTA models. Then additionally, we present a model improved for high-resolution prediction, which achieves better results on HRRR data and our extreme weather dataset. This serves as strong a baseline model for future predictions on HRRR data and its extreme events. > **Evaluate HR-Heim on a similar extreme weather dataset** **A**: Thank you for your suggestion. The experiment of HR-Heim at different lead times are added to paper at the end of Section 4.3 in blue, showing its consistent superior performance. HRRR data offers significantly higher resolution than ERA5, while the dataset you referenced is built on ERA5. In contrast, HR-Heim is specifically optimized for high-resolution prediction on HRRR and trained on HRRR data. Nonetheless, our dataset addresses the gap in high-resolution datasets for extreme weather, with its primary contribution being its application for model evaluation.

Thanks!

Thank you for taking the time to reassess your concerns regarding our work, we are grateful for your constructive engagement!

Response to Reviewer kLu6 [Part 1]

Thank you for your review and we understand your feedback. It seems there may have been some misunderstanding regarding the usage and purpose of our dataset, as well as the settings of the HRRR data. > **Concerns about the significance of our work** **A**: - **Clarification of Dataset Purpose**: We understand that you may view our work as a dataset for training models to better predict extreme events. However, our work is actually a dataset for evaluating the performance of models on recorded extreme events based on HRRR data. We understand your concern about the uncertainty of extreme weather and may think that our extreme dataset is built on a simulator. However, our dataset is systematically constructed from recorded extreme events and is primarily intended for evaluation. We have also stated in the future work in our paper that it is only a possibility for future models to fine-tune on our dataset to improve their performance. - **Contributions and Significance**: The primary goal of this work is to address the gap in high-resolution datasets for extreme weather evaluation, encompassing comprehensive event types. We provide such a dataset to evaluate the ability of SOTA models on recorded extreme events. We aim to highlight the significance of extreme weather prediction for future research, demonstrate the substantial room for improvement in the performance of current SOTA models. Additionally, we present a model improved for high-resolution prediction, which achieves better results on HRRR data and our extreme weather dataset. This serves as a baseline model for future predictions on HRRR data and its extreme events. - **Correctness of our Dataset and NWP Model Usage**: You might have been misled by the HRRR dataset settings. The analysis at lead time 0 (often referred to as f00) is a blend of the model's background (a short-term forecast from the previous cycle) and recent observations, integrated through data assimilation techniques. This process incorporates various observational data, such as from aircraft, radar ..., to produce an accurate representation of the current atmospheric state. Therefore, while the f00 output is heavily informed by actual observations. For lead times greater than 0 (e.g., f01, f02), the outputs are forecasts generated by the model based on the initial analysis [3]. And our dataset is built on data at lead time 0, the predictions of NWP uses lead time at least 1, which enables fair comparison to other models. - **Importance of Extreme Weather Nowcasting**: We want to state that extreme weather nowcasting is also a crucial task in practice, such as work NowcastNet[1] and DGMR[2] published on Nature. We have already considered the unusual atmosphere states before and after the extreme events, therefore we incorporate several hours of atmospheric state data before and after the recorded events to include the comprehensive progression of an extreme event. On top of that, our open-source code is designed to allow users to adjust the temporal dimension, enabling arbitrary lead-time predictions or extending the atmospheric state analysis before and after the event. This flexibility supports both nowcasting and longer forecasting evaluation. > [1] Zhang, Y., Long, M., Chen, K. et al. Skilful nowcasting of extreme precipitation with NowcastNet. Nature 619, 526–532 (2023). [2]Ravuri, S., Lenc, K., Willson, M. et al. Skilful precipitation nowcasting using deep generative models of radar. Nature 597, 672–677 (2021) [3] How do I download 15-minute HRRR data: https://github.com/blaylockbk/Herbie/discussions/67#discussioncomment-2537530 > **Question about model training** **A**: Thank you for your question. All models (Pangu, Fuxi, and our HR-Heim) were trained on HRRR data spanning the U.S. from January 2019 to June 2020, from scratch. They were trained under identical parameters and same level of model parameters, and no hyperparameter tuning was applied to HR-Heim. Furthermore, none of these models were fine-tuned on our HR-Extreme dataset, ensuring a fair basis for comparison and evaluation, which means none of the models have seen the extreme event in training. This clarification has been added at the beginning of Section 4.3 > **Concerns about Figure 4** **A**: The intention is not to compare the models’ abilities to predict extreme events but to validate the dataset’s capability in identifying high-error regions associated with extreme weather. The purpose of Figure 4 is to demonstrate that our dataset effectively captures the regions with the greatest prediction loss for each model. These regions correspond to extreme weather events included in our dataset, which was constructed using recorded data and manual filtering. We have also added additional analysis of physical variables in appendix A.1.

Response to Reviewer Jbrg [Part 1]

Thank you for very much your positive feedback ! We truly appreciate your insightful understanding and recognition of the key contributions of our study in terms of significance, comprehensiveness and open-source code and dataset! We have revised our paper according to your suggestions and provide our responses to address your concerns below. > **Question about clustering method** **A**: Thank you for your question. We already include the detailed introduction of the clustering method we used (DBSCAN) is in the end of the part "NOAA Storm Prediction Center" in section 3.2, but to make it clearer, we improve our writings for it and also put it here for your convenience. "After extensive case studies, we determined that DBSCAN (Ester et al., 1996) is most suitable for this task, as illustrated in Figure 1. For each timestamp, user reports are treated as 2D points based on normalized latitude and longitude on the x and y axes. DBSCAN identifies clusters based on point density, forming a cluster if there are enough points in close proximity. We carefully tune the hyperparameters of DBSCAN to create more intuitive clusters and to filter out noisy points more accurately as shown in Figure 1. Noisy points are filtered out because they likely represent minor events or errors that are not significant enough to warrant creating a separate cropped area for evaluation." > **Question about model training** **A**: Thank you for your question. All models (Pangu, Fuxi, and our HR-Heim) were trained on HRRR data spanning the U.S. from January 2019 to June 2020, from scratch. They were trained under identical parameters and same level of model parameters, and no hyperparameter tuning was applied to HR-Heim. Furthermore, none of these models were fine-tuned on our HR-Extreme dataset, ensuring a fair basis for comparison and evaluation. This clarification has been added at the beginning of Section 4.3 in blue. > **Concern about clustering results and more analysis** **A**: Thank you for your question. We have verified our clustering results for finding the area extreme event by intensive case studies when we built our dataset. We carefully tuned the hyperparameters of it to make the results consistent with the recorded information. We have also added an additional statistical analysis of physical variables in different extreme weather events in appendix A1 showing the distinct characteristics of extreme events compared to normal cases. > **Question about the sentence: "types of events without specific ranges or those not related to obvious variations in feature map predictions"** **A**: Thank you for your question. This sentence describes our data cleaning process for entries from the NOAA Storm Events Database. To make it clearer, we have reorganize the sentences as the following and fixed it in Section 3.2 in orange . While most events include information on location and spatial range, some lack these critical details. Additionally, certain event types, such as avalanches and high surf, do not significantly impact the physical variables predicted by the models. To ensure greater accuracy, we have filtered out these events. >**Can data from different sources be used separately?** **A**: Yes! The index file generated along with the dataset allows users to easily retrieve information on event types, time spans, and locations for each event from any data source. However, only use a single data source would be incomplete, as the three data sources that we utilize each one contributes unique information on different types of extreme events with varying extents. (more details in Section 3.2)

Response to Reviewer wJpu [Part 1]

Thank you for your positive comments, and thank you for your recognition of the paper’s importance, comprehensiveness and open-source code and dataset! We have revised our paper according to your suggestions and provide our response to address your concerns below. > **Question about how HR-Heim is Trained** **A**: Thank you for your question. All models (Pangu, Fuxi, and our HR-Heim) were trained on HRRR data spanning the U.S. from January 2019 to June 2020, from scratch. They were trained under identical parameters and same level of model parameters, and no hyperparameter tuning was applied to HR-Heim. Furthermore, none of these models were fine-tuned on our HR-Extreme dataset, ensuring a fair basis for comparison and evaluation. This clarification has been added at the beginning of Section 4.3 in blue. > **Questions about more metadata in dataset** **A**: Thank you for your question. This information is already included in the index file. Alongside the dataset generated by our code. The index file contains details on location, range, type of extreme weather, and time span by integrating information from three data sources. Users can easily convert this information from the index file to longitude and latitude coordinates with our open-source code. We have added this clarification at the beginning of Section 3.3 in blue. > **Questions about more analysis and comparison to ERA5** **A**: Thank you for your suggestion. We have also added an additional statistical analysis of physical variables in different extreme weather events in appendix A1 showing the distinct characteristics of extreme events compared to normal cases. In terms of ERA5, we have considered ERA5 in our research, however, it is not suitable for building our extreme dataset. ERA5 data has a resolution of 31 km, while HRRR data provides a finer resolution of 3 km. Due to this difference, many extreme events with limited spatial range are more clearly represented in HRRR data but appear vague or may be entirely absent in ERA5. It is more accurate and beneficial for future work to develop our dataset using HRRR data.