Precipitation Downscaling with Spatiotemporal Video Diffusion

Summary

This paper presents a novel framework for spatio-temporal precipitation downscaling, comprising two modules: a deterministic downscaling module and a diffusion module. The model is able to outperform the SOTA models, especially in extreme events and in mountainous areas.

Strengths

1. Multiple losses, such as PE and EMD, are used to measure the effectiveness of the results. At the same time, the discussion of the trade-off between realism and bias is novel and informative, using MSE to represent the average accuracy of predictions and PE to represent the model's ability to reproduce extreme events. 2. The model outperforms six strong super-resolution baselines and can be established as a new standard for data-driven precipitation downscaling.

Weaknesses

Experiments are insufficient. Such as the effectiveness of the sharing features across the modules is not proven by ablation experiments.

Questions

1. In line 121, “bicubic interpolation” does not have a explanation. Can you explain it? 2. In line 121, what is ”pixelated features” ? 3. Is the input, additional climate states, the L1 data or L2 data?

Rating

5

Confidence

5

Soundness

2

Presentation

3

Contribution

2

Limitations

Is this work of practical value? The authors acknowledge that switching to a different dataset requires retraining due to differences in data distribution. However, can the model be trained if high-resolution ground truth images are not available? If high-resolution images are available, does it make sense to reduce their resolution before training the model?

Summary

The method proposes a diffusion model to statistical downscale precipitation. The model requires a combination of high resolution and low resolution video data for training (a common scenario in weather and climate modeling). The diffusion model takes in a video of low resolution atmospheric variables and outputs a sample video of high-resolution precipitation. In experiments, the model is trained on an ensemble of year-long runs from the standard US forecasting system (FV3GFS). Comparisons are made to deterministic downscaling models. The models are evaluated using mean squared error (MSE), continuous rank probability score (CRPS), earth mover distance, 99.999th percentile error (PE), and spatial autocorrelation error (SAE). The results confirm the advantage of using a probabilistic method in estimating the risk of extreme events.

Strengths

- The use of machine learning to downscale atmospheric variables, particularly precipitation, is a hot topic in weather forecasting and climate modeling. This paper will be of interest to the community. - Nice ablation studies showing the importance of the temporal dimension and additional atmospheric variables.

Weaknesses

- There have been a number of papers using diffusion models for downscaling (Hatanaka, et al. 2023 for solar irradiance) and nowcasting (Gao, et al. 2023 and Yu, et al. 2024 for precipitation). These are missing from the related work section. - The use of transformer-based models is not justified. It seems likely that vision transformer models have a poor inductive bias for this task, which probably has a lower degree of long-range spatiotemporal dependency than in natural images/video. The authors mention "key adaptations to the attention mechanism" but these appear to be simplifications that reduce computation and increase the locality bias --- have the authors tried removing attention all together? - Some typos that could be caught with a spell checker. - The baselines used for comparison are weak. They are all deterministic, so it is expected that they will fail to capture the extremes. I don't think it is necessary to have comparisons, but I think this point could be made more clearly in the text. - In experiments, only 10 samples are taken from the diffusion model. This seems small in the context of estimating the risk of extreme events. - Some clarifications would be helpful (see questions below).

Questions

- 280: For an annual average, the diffusion model isn't really necessary. Wouldn't any of the deterministic statistical downscaling models would perform just as well? This should be clarified. - 294: It was unclear from the text why additional sampling steps in the STVD increases MSE. I imagine this is because the deterministic downscaling model predicts the mean of the possible rainfall (low MSE) and sampling from the distribution of residuals with the diffusion model will almost certainly increase this. So more sampling steps means a better sample of the residual, which means a higher MSE?

Rating

7

Confidence

5

Soundness

3

Presentation

3

Contribution

3

Limitations

- As the authors note, the model is demonstrated on simulation data where high-resolution video inputs are provided. In many applications of statistical downscaling, the input and output are from different sources (e.g. observed high-res vs. GFS model output or reanalysis), and there may be inconsistencies. So the experiments done here are on "clean" data, and further experiments are needed to evaluate the method on applications where it will be useful.

Summary

This paper extends video diffusion model to precipitation super-resolution, where a deterministic downscaler is used to produce initial results and a temporally-conditioned diffusion model is utilized to refine previous coarse results. By combing deterministic and statistical downscaling models, "mode averaging" problems are obviously alleviated. Experimental results demonstrates its effectiveness.

Strengths

1. The paper is well organized, the motivations and method details are clearly described. 2. The explanation of professional terms is very good, so that researchers in other fields can easily read the paper. 3. It is very reasonable to predict the low-frequency part with a deterministic model and then generate the high-frequency residual with a statistical model. 4. The comparisons with other methods in experiment parts seems sufficient.

Weaknesses

There are some statements, method and experiment details remain to be clear. 1. Is there any design that guarantees that the output high-resolution frames are smooth over the time series? 2. For the high-frequency prediction part, if given the same conditions but different sampling noise, will output completely different results? 3. The authors state that the generative models can capture multimodal conditional distributions and alleviate underestimation of extreme precipitation. Could the authors show experimentally that their approach is better at modeling extreme precipitation than traditional supervised methods?

Questions

Please refer to Weaknesses part.

Rating

5

Confidence

2

Soundness

3

Presentation

4

Contribution

2

Limitations

The authors clearly point out the limitations of their work and the negative social impact after Conclusion section.