Doubly Robust Proximal Causal Learning for Continuous Treatments

Response to Reviewer eqxp

We appreciate your efforts in reviewing our paper and constructive comments. We have revised our manuscript based on your advice. We address your concerns in the following. **Q1.** The paper’s rationale for the calculation of the influence function appears to be lacking, as it directly selects a specific submodel. It might be more appropriate to refer to the function derived in this paper as the efficient influence function, given that the previous doubly robust estimator for binary treatment is efficient. **A.** While the DR estimator for binary treatment is shown to be the effective influence function, it is unclear whether it still holds for continuous treatments without explicit computation. To verify, we calculate the influence function. **Q2.** Why does the projected residual mean squared error make sense? Does it mean $\hat{q}$ is consistent for $q_0$? **A.** The projected residual mean squared error measures how much $\hat{q}$ violate Eq. (3). This performance metric holds significant prominence in the theoretical analysis of minimax problems, as noted in references [1, 2, 3]. When the measures of the ill-posedness of inverse problems are bounded, $\hat{q}$ is consistent for $q_0$. Please refer to Remark 6.3 for details. **Q3.** Please make the motivation clear about why this paper does not focus on the inference issue about the analysis of asymptotically distribution for a causal effect. **A.** Currently, we cannot obtain the asymptotic normality due to the error introduced in kernel approximation. With this error, we can show in Theorem E.9 that our estimator is $n^{2/5}$-consistent, while the asymptotic normality means $\sqrt{n}$-consistent. Besides, according to [4], since the estimand is non-regular, therefore it may not enjoy the properties of $\sqrt{n}$-consistent and asymptotically normality. We illustrate this point through an empirical study in Appendix E.5. **Q4.** The empirical coverage probability is not given. MSE only contains both bias and variance terms, which may not display the true statistic estimation accuracy and precision. **A.** Thank you for your suggestions. We provide in Theorem E.9 that our estimator is $n^{2/5}$-consistent, which means with high probability, the error is $O(n^{-2/5})$. **Q5.** This paper seems to consider the high-dimensional setting in experiments, what about theoretical properties? **A.** In this context, we follow the setting in [5,6], in which the term "high-dimensional" merely indicates a scenario with relatively more covariates compared to those in section 7.1.1, without implying that the number of samples is smaller than the number of features. Therefore, our theory still applies to this case. [1] Dikkala, Nishanth, et al. "Minimax estimation of conditional moment models." Advances in Neural Information Processing Systems 33 (2020): 12248-12262. [2] Ghassami, AmirEmad, et al. "Minimax kernel machine learning for a class of doubly robust functionals with application to proximal causal inference." International Conference on Artificial Intelligence and Statistics. PMLR, 2022. [3] Qi, Zhengling, Rui Miao, and Xiaoke Zhang. "Proximal learning for individualized treatment regimes under unmeasured confounding." Journal of the American Statistical Association (2023): 1-14. [4] Colangelo, Kyle, and Ying-Ying Lee. "Double debiased machine learning nonparametric inference with continuous treatments." arXiv preprint arXiv:2004.03036 (2020). [5] Xu, Liyuan, Heishiro Kanagawa, and Arthur Gretton. "Deep proxy causal learning and its application to confounded bandit policy evaluation." Advances in Neural Information Processing Systems 34 (2021): 26264-26275. [6] Kompa, Benjamin, et al. "Deep learning methods for proximal inference via maximum moment restriction." Advances in Neural Information Processing Systems 35 (2022): 11189-11201.

Response to Reviewer sZEk

We appreciate reviewer sZEk for the time and effort in reviewing our paper. We address your concerns below. **Q1.** I believe that the statement of Theorem 4.5 is incorrect. More precisely, the influence function of $B(a)$ does not exist, meaning that $B(a)$ is not pathwise differentiable. However, $B(a,P^{\varepsilon,h_{bw}})$ is pathwise differentiable, indicating that the influence function does exist. Therefore, to accurately state this, the term `lim' should be removed and the statement should be made with for any $h_{bw}>0$. **A.** While $\beta(a,P^{\varepsilon,h_{bw}})$ is pathwise differentiable, its influence function is not of our interest. Our goal is to obtain the influence function for $\beta(a)$. To obtain this, we take the gradient with respect to $\varepsilon$ and let $h$ go to 0. Such a treatment has been similarly adopted in [1]. **Q2.** A practical guide is needed to solve the optimization problem in Equations (8,9) and apply these methods in practice. **A.** Equations (8,9) are a minimax optimization problem, and we can solve it in different ways, depending on what function space class we use. For example, We can parameterize $q$ (resp. $h$) and $m$ (resp. $g$) as reproducing kernel Hilbert space (RKHS) with kernel function or neural networks. For the former, we derive their closed solutions in Appx. F. In the latter case, we can employ Generative Adversarial Networks [2]. We have appended this discussion to make it clearer. **Q3.** I understand that $q_0$ represents the maximum value for the equation in Lemma 5.1. However, I am unsure about the connection between Lemma 5.1 and Equation (8). In other words, why do we need to minimize the empirical quantity for the equation in Lemma 5.1? **A.** We have modified the description of Lemma 5.1 and added a paragraph to discuss its connection to Eq. (3) and Eq. (8). Simply speaking, We first show that solving $q_0$ from data is equivalent to minimizing $\mathcal{L}_q(q;p):=\mathbb{E} [ \left( \mathcal{R} _q\left( q,p \right) \right) ^2 ]$ over $q$, where $\mathcal{L}_q(q;p)$ is equivalent to the maximization form in Lemma 5.1. [1] Hidehiko Ichimura and Whitney K Newey. The influence function of semiparametric estimators. Quantitative Economics, 13(1):29–61, 2022. [2] Goodfellow, Ian, et al. "Generative adversarial nets." Advances in neural information processing systems 27 (2014).

Response to Reviewer jsr3

Thank you for the positive assessment and valuable suggestions on our paper. We address your questions below. **Q1.** Recommending a detailed comparison to the existing literature in the appendix --- how is this method novel and why this novel kernel modification is necessary to handle the pitfalls in the current literature? **A.** We have claimed in our **contribution** and the second but last paragraph in the introduction that our method for the first time can estimate the causal effect over continuous variables, even if the unboundedness assumption is violated. To better stand out our uniqueness, we have appended a related work section in **Appx. B** that gives a comprehensive introduction to related works and how our method differs from it. We use kernel modification because its kernel approximation is consistent (Theorem 4.2); more importantly, it can approximate the influence function according to Theorem 4.5. **Q2.** More numerical experiments. **A.** We conduct experiments on four additional settings, encompassing three synthetic data with different data-generating mechaninisms, as well as a time-series forecasting setting. Besides, we compare with two additional DL-based baselines *MINMAX* and *NMMR*. Results in Tab. 1 and Tab. 3 (Appx. G) suggest that our methods can outperform others in these settings. For completeness, we attach the table of additional experiments in the Appx. G to the rebuttal console: | Dataset | | Size | PMMR | KPV | DFPV | MINIMAX | NMMR-V | POR | PKIPW | PKDR | |-------------|------------|-------|-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------| | | | "Hu et al. | Scenario 1 | 1000 | 0.25±0.05 | 0.25±0.05 | 0.59±0.35 | 1.45±1.32 | 0.17±0.09 | 0.16±0.23 | 0.29±0.12 | **0.16±0.22** | | (2023)" | Scenario 2 | 1000 | 0.16±0.02 | 0.16±0.02 | 0.22±0.17 | 0.88±0.29 | **0.05±0.04** | 0.08±0.07 | 0.15±0.06 | 0.07±0.06 | | | Scenario 3 | 1000 | 0.10±0.02 | 0.10±0.02 | 0.28±0.39 | 0.45±0.29 | 0.21±0.10 | 0.22±0.20 | **0.09±0.03** | 0.21±0.19 | | Time series | | 500 | 0.11±0.04 | 0.13±0.12 | 0.20±0.14 | 0.21±0.09 | 0.18±0.12 | 0.10±0.14 | 0.21±0.05 | **0.09±0.12** | | | | 1000 | 0.10±0.05 | **0.08±0.07** | 0.16±0.21 | 0.22±0.06 | 0.18±0.10 | 0.12±0.12 | 0.20±0.06 | 0.12±0.10 |