Response to Reviewer KUb5
Thank you for your reply! We are happy that we have addressed your concerns sufficiently!
We also welcome the opportunity to answer your remaining question about the empirical Wasserstein distance.
The Wasserstein distance is a commonly used metric in machine learning [1,2,3] for quantifying the distance between two probability distributions based on the concept of optimal transport. In our work, we use it to evaluate the model performance in learning the potential outcome distributions. Thus, this allows us to move _beyond evaluating point estimates_ to the _full distribution_.
The $k$-Wasserstein distance (for any $k \geq 1$) between two distributions $\mu_1$ and $\mu_2$ is
$W^k\left(\mu_1, \mu_2\right)=\left(\int_0^1\left|\mathbb{F}_1^{-1}(l)-\mathbb{F}_2^{-1}(l)\right|^k \mathrm{~d} l\right)^{1 / k}$
where $\mathbb{F}_1^{-1}(l)$ and $\mathbb{F}_2^{-1}(l)$ are the quantile functions (inverse cumulative distribution functions) of $\mu_1$ and $\mu_2$ for quantile $l$, respectively.
In practice, Wasserstein distance is estimated by the empirical Wasserstein distance based on two sets of finite samples, i.e.,
$\hat{W}^k\left(\mu_1, \mu_2\right)$ $=\left(\frac{1}{n} \sum_{i=1}^n\left\|X_i-Y_i\right\|^k\right)^{1 / k}$,
where $X_1, \ldots, X_n$ are the samples from $\mu_1$ and $Y_1, \ldots, Y_n$ are the samples from $\mu_2$, and we set $k=1$ in our experiments, following [1].
(3) We use empirical Wasserstein distance to evaluate our model performance on learning the potential outcome distributions, where _lower_ distances imply _better_ performance. Here, our method gives the **best** results and outperforms the baselines by a clear margin (see **Table 1 / Rebuttal PDF**). This demonstrates the strong performance of our model not only on point estimates, but also on estimating the _full distribution_.
Thank you again for reading our rebuttal! We hope that the above results answer all your questions.
We would greatly appreciate it if you would raise your score accordingly. Thank you. And do let us know if you have further questions -- we will do our best to answer them promptly.
Thanks again for reviewing our submission!
Reference:
[1] Maeda T, Ukita N. Fast inference and update of probabilistic density estimation on trajectory prediction. International Conference on Computer Vision. 2023
[2] Melnychuk V, Frauen D, Feuerriegel S. Normalizing flows for interventional density estimation. International Conference on Machine Learning. 2023.
[3] Letzelter V, Perera D, Rommel C, et al. Winner-takes-all learners are geometry-aware conditional density estimators. International Conference on Machine Learning. 2024.