We appreciate the reviewer for your reading and reply. We apologize for not adequately explaining the comment on the out-of-distribution detection. Let us elaborate on this point a bit more. There are two different scenarios for detecting out-of-distribution data, as follows:
1) The goal is to check whether the joint distributions are identical, i.e., whether $(X_{test}, Y_{test})$ is from the same distribution as $(X_{train}, Y_{train})$. To this end, we can estimate the aleatoric uncertainty $\pi_{Y \mid X}$ and check whether $Y_{test}$ falls into the highest density region of $\pi_{Y \mid X_{test}}$ [1]. If the answer is no, then with high probability, $Y_{test}|X_{test}$ is out-of-distribution, and thus $(X_{test}, Y_{test})$ is also out-of-distribution. This approach considers the conditional distributions of Y given X.
This is the scenario we referred to in the rebuttal, which is indeed related to our problem setting. In particular, our problem aims to quantify the uncertainty in the obtained model $\hat{h}(x)$ against the target best model $h^*(x)$ (with a fixed $x$). We do so by providing a confidence interval around $h^*(X_{test})$ (not $Y_{test}$), because we focus on the quantification of epistemic uncertainty. On the other hand, $Y_{test}$ typically contains noise in regression tasks (i.e., the aleatoric uncertainty is not zero), and to carry out the out-of-distribution detection scheme, the construction of the highest density region needs to consider the aleatoric uncertainty.
2) The reviewer probably means the second scenario that has been widely studied in classification: The goal is to check whether the marginal distributions are identical, i.e., whether $X_{test}$ is from the same distribution as $X_{train}$ (typically given some conditions, such as for a certain class). To our knowledge, this task generally requires estimating the distribution of $X_{train}$ or its features (extracted from the neural network), and then checking whether $X_{test}$ fits. For instance, [2,3] use information from the likelihood function based on the estimated density function of $X_{train}$ from generative models, [4,5] check the energy score or softmax score from the distribution of the final layer's logits, [6] aggregates the p-values from the distribution of intermediate layers’ features. These works are related to quantifying the distribution of $X_{train}$ or its features (logits can also be viewed as condensed features), while our work has a different focus: The epistemic uncertainty is regarding the response $\hat{h}(x)$ while $x$ is a fixed test point (not random), and the randomness is from the training data and training procedure (different in-distribution training data and training procedures lead to different responses $\hat{h}(x)$).
References:
[1] Hyndman, Rob J. "Computing and graphing highest density regions." The American Statistician 50.2 (1996): 120-126.
[2] Nalisnick, Eric, et al. "Do Deep Generative Models Know What They Don't Know?." International Conference on Learning Representations. 2019.
[3] Grathwohl, Will, et al. "Your classifier is secretly an energy based model and you should treat it like one." International Conference on Learning Representations. 2020.
[4] Liu, Weitang, et al. "Energy-based out-of-distribution detection." Advances in neural information processing systems 33 (2020): 21464-21475.
[5] Hendrycks, Dan, and Kevin Gimpel. "A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks." International Conference on Learning Representations. 2017.
[6] Haroush, Matan, et al. "A Statistical Framework for Efficient Out of Distribution Detection in Deep Neural Networks." International Conference on Learning Representations. 2022.