Sharp Bounds for Generalized Causal Sensitivity Analysis

Causal inference from observational data is crucial for many disciplines such as medicine and economics. However, sharp bounds for causal effects under relaxations of the unconfoundedness assumption (causal sensitivity analysis) are subject to ongoing research. So far, works with sharp bounds are restricted to fairly simple settings (e.g., a single binary treatment). In this paper, we propose a unified framework for causal sensitivity analysis under unobserved confounding in various settings. For this, we propose a flexible generalization of the marginal sensitivity model (MSM) and then derive sharp bounds for a large class of causal effects. This includes (conditional) average treatment effects, effects for mediation analysis and path analysis, and distributional effects. Furthermore, our sensitivity model is applicable to discrete, continuous, and time-varying treatments. It allows us to interpret the partial identification problem under unobserved confounding as a distribution shift in the latent confounders while evaluating the causal effect of interest. In the special case of a single binary treatment, our bounds for (conditional) average treatment effects coincide with recent optimality results for causal sensitivity analysis. Finally, we propose a scalable algorithm to estimate our sharp bounds from observational data.

Paper

Similar papers

Peer review

Reviewer GZoa6/10 · confidence 4/52023-06-19

Summary

The authors generalize a class of causal sensitivity models that includes the traditional MSM, the continuous-treatment CMSM, and the longitudinal (time-varying treatment) LMSM. They show how to compute sharp bounds for the causal estimands by taking inspiration from recent work. Their general framework also allows mediation analysis. They provide an algorithm for computing these bounds.

Strengths

The method is solid and explained well. Figure 2 is nice. Mediation analysis is an important contribution to causal sensitivity models. A broader understanding of sensitivity models is always valuable and the effort to generalize is a good one.

Weaknesses

My main concern is that this generalization that unifies the MSM, CMSM, and LMSM is not very useful. The weighting function seems a bit contrived. It is necessary because the CMSM and LMSM do not use the nominal propensity at all, but the MSM does. Isn't it strange to ignore the nominal propensity and give the same bounds to all potential outcome distributions? Shouldn't the observed confounding inform the unobserved confounding? A recent alternative to the CMSM is the $\delta$MSM [arXiv:2204.11206] that takes a different approach and appears to perform better. The authors could discuss alternative models like this one or at least keep them in mind when considering general classes of sensitivity models. The clever approach to sharp partial identification is not novel [see for instance arXiv:2304.10577]. The benefit of the mediation analysis is not really made clear in the results of this submission. In terms of results, the authors employ a purely synthetic benchmark. The real-world data are interesting but they lack a ground truth. The authors could include some well-known semi-synthetic benchmarks like IHDP and induce hidden confounding by hiding some of the observed confounders. The authors do not compare with previous methods in their benchmark except for Table 1, where they use a custom weighting function to beat the older CMSM algorithm. That is not very convincing. Also Table 1 should at least be discussed more. The results for the weighted CMSM seem conflicting: tighter bounds but worse coverage? On a lesser note, the technical details are a bit dense.

Questions

Could you support your reasoning for why this weighting function is a natural interpretation of the more specific sensitivity models? How is it helpful and how can I use it in newer settings? Why should it be set to zero in some cases?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

Limitations are discussed a bit but they do not address potential societal impacts. It is debatable if that is necessary for this kind of work, but I think pitfalls of these kinds of sensitivity analyses should be discussed.

Reviewer H5426/10 · confidence 3/52023-07-05

Summary

This paper is about sensitivity analysis (SA) of causal queries in SCMs. In practice, given a causal query and a set of models, the goal is to compute a query's lower and upper bounds. The authors first derive a class of models to be used for SA and show how this extends existing models. An algorithm to obtain the bounds is such cases is derived. (After the rebuttal, I decided to raise the rating of the paper from 4 to 6)

Strengths

The technical results are sound and non-trivial. The experiments show good bounds obtained in this way.

Weaknesses

The literature on partially identifiable queries is ignored. In particular existing techniques for bounding such queries are not considered.

Questions

Would it be possible to compare the present method against algorithms for the bounding of non-identifiable queries? E.g. Zhang and Bareinboim, Duarte et al., and Zaffalon et al. worked in this direction in the last two years. I think the sensitivity analysis the authors consider is a heuristic approach to the same problem. If this is true, not having a comparison against these methods is a serious issue. I also believe that the authors would take advantage of the literature about so-called "imprecise probabilities" and "credal networks" as these models implement the kind of sensitivity analysis of interest for the authors. In particular, the paper "Structural Causal Models Are (Solvable by) Credal Networks" might be a helpful reading.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

2 fair

Limitations

I don't see specific issues in this direction.

Reviewer taFV8/10 · confidence 3/52023-07-05

Summary

The authors propose a unified framework for causal sensiitivity analysis under unobserved confounding that generalizes the Marginal Sensitivity Model (MSM). They derive sharp bounds for a diverse set of causal effects such as the CATE, mediation and path analysis effects, and distributional effects. The framework is applicable to discrete, continuous, and time-varying interventions. They offer, to my knowledge, a novel interpretation of the marginal sensitivity model via SCM. They show that in the case of binary treatments, their derived bounds coincide with the optimality result of Dorn and Guo 2023. They provide a closed form solution to and algorithm to estimate the bounds and show empircally that it improves over line search methods (a formal complexity analysis would be nice).

Strengths

This paper makes several novel contributions to an active area of study in causal machine learning, which are listed above in the summary. They provide theoretical and experimental evidence supporting their claims. They do a good job presenting and comparing to the related work. An exceptionally well writen and organized paper given the complexity of the subject matter.

Weaknesses

## I have one primary comment. I think there may be a step missing in the special cases proofs of Appendix C. I appreciate how the authors have utilized SCM to define the GMSM. But, in defining the MSM, CMSM, and LMSM in terms of hidden confounders $u$, it seems that there is a step missing from how these models are originally defined. Namely, they are defined with respect to potential-outcomes / counterfactuals $Y_{t}$ and the conditional independence relation: $Y_t \perp T \mid X$. For example, it's not obvious to me how you move from $P(a \mid x, y_t)$, to $P(a \mid x, u)$. Given that you show equivalence in terms of $P(a \mid x, u)$, I think it is important to be explicit here. If this is resolved, and I admit that this could be completely trivial and I just don't see it, I would happily increase my score. If it cannot be resolved, I would suggest removing these claims and my score would remain the same. ## I have a few minor comments. First, the last two paragraphs of the introduction can be streamlined as the contributions paragraph essentially reiterates the points of the paragraph starting on line 45. I like both styles, with slight preference for the contribution format. There is a recent paper proposing a marginal sensitivity analysis for continuous treatments that could be added to the related works. Line 70 "... when while ..." seems to be a typo

Questions

How does one show that $P(a \mid x, y_t) = P(a \mid x, u)$?

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

4 excellent

Contribution

4 excellent

Limitations

Yes

Reviewer 9HdS6/10 · confidence 4/52023-07-06

Summary

The authors study the problem of bounding a given causal effect. To this end, they propose a generalized marginal sensitivity model (GMSM) that is applicable to multiple discrete, continuous , and time-varying treatments. They also present a new interpretation of the partial identification.

Strengths

The proposed GMSM model generalizes the previous models and leads to sharp bound for certain causal effects with certain causal graphs. The bounds depend on observed variables and thus can be estimated. Except some minor ambiguities (see below), the paper is well-written.

Weaknesses

Although the proposed model generalized the previous models but the presented theoretical results hold under specific graphical constraints e.g., no confounder between M and outcome, no hidden confounders between X and $\{A,M,Y\}$. The other weakness about the results is its generalizability to arbitrary causal graphs. Based on the presented proofs in the appendix, it is not clear how this results can be generalized by relaxing the assumptions. There is ambiguity about the notation of U. Does $U_Y$ denote the unobserved exogenous variable for Y in the definition of SCM or is it a hidden confounder between Y and A? Is it possible to have hidden confounders among the mediators (e.g., $M_1$ and $M_2$)? The explanation below (3) says “If U_w has no effect on A, Eq. (3) holds with $s^-_W (a, x) = S^+_W (a, x) = 1$”. But it seems that (3) encodes the effect of A on $U_W$.! In (35) in the Appendix, what is U exactly? Does the setting imply that u and x are independent, i.e., $p(u|x)=p(u)$?

Questions

See above.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

3 good

Contribution

3 good

Limitations

The authors addressed the limitations.

Reviewer taFV2023-08-11

Thank you for your response. You have addressed my concerns and I would like to raise my score to an 8.

I have read your response and I appreciate the efforts you have made to address my concerns. I'm particularly impressed by the lemma you have provided, as the relationship there is something that has bothered me for some time. Trusting that the action points will be incorporated into the camera ready version, I would like to increase my score to an 8.

Authorsrebuttal2023-08-14

Thank you for acknowledging our response and for providing swift feedback. We are happy that we were able to resolve the ambiguity regarding the sensitivity model definition. We will incorporate all action points as promised. Thank you also for your willingness to increase your score to an 8. We saw in the system that the number did not change yet, and, for that reason, we wanted to simply follow back if this is still a to-do or if this is something with OpenReview. If you have any further questions or requests, please let us know.

Reviewer H5422023-08-16

Thanks for the clarification on the CSA vs. CPA thing. This motivates me to raise my score.

I appreciate the clarification about the difference between CSA and CPA provided by the authors—many thanks for that. I have experience with CPA but not with CSA. This motivated my question and doubts about the paper. The rebuttal affects my evaluation of the paper, and I am happy to move towards a positive recommendation. Regarding the different points raised by the authors in their rebuttal, I am not very convinced by the argument in the item starting with "There is no free lunch in causal inference". I don't believe that the fact that most of the papers on CPA cope with discrete (endogenous) variables reflects a necessary additional assumption. This is only related to the existing works starting from the more straightforward discrete case. Still, similar techniques will undoubtedly be explored soon also for continuous variables.

Authorsrebuttal2023-08-16

Thank you for your response and raising your score

Thank you for acknowledging our response and for raising your score. Please allow us to clarify our argument "There is no free lunch in causal inference". What we meant to say is, that in the standard CATE setting, CPA corresponds to setting $\Gamma \to \infty$ for our bounds. This is because we obtain our bounds by optimizing over all possible SCMs that are compatible with (i) the causal graph, (ii) the observed data distribution, and (iii) the sensitivity constraints. When setting $\Gamma \to \infty$, we ignore (iii) and only constrain our class of SCMs by (i) and (ii), thus performing CPA. **Hence, any other CPA approach for tighter bounds would provably require stronger assumptions.** For example, there are existing CPA approaches that yield tighter bounds by exploiting valid instrumental variables (see e.g., [3, 4]). In the following, we characterize our (w.l.o.g. upper) bounds for $\Gamma \to \infty$ in the standard CATE setting: Observed covariates $X$, binary/ continuous treatment $A$, and continuous outcome $Y$. We are interested in the causal query $Q(x, a, \mathcal{M}) = \mathbb{E}\left[Y \mid x, do(A = a)\right]$ (Example 1 from our paper). Our upper bound is $Q^+ = \int_{\ell}^{F^{-1}(c_Y^+)} \frac{y}{s_Y^+} \mathbb{P}(y \mid x, a) dy + \int_{F^{-1}(c_Y^+)}^{u} \frac{y}{s_Y^-} \mathbb{P}(y \mid x, a) dy $, where $c_Y^+ = \frac{\Gamma}{1 + \Gamma}$ and $\ell, u$ are the lower/ upper support points of the distribution $\mathbb{P}(y \mid x, a)$, respectively. 1) For binary treatment $A \in \{0, 1\}$, using the MSM we obtain \begin{equation} Q^+ = \int_{\ell}^{F^{-1}(\frac{\Gamma}{1 + \Gamma})} y \left((1 - \Gamma^{-1}) \mathbb{P}(a \mid x) + \Gamma^{-1} \right) \mathbb{P}(y \mid x, a) dy + \int_{F^{-1}(\frac{\Gamma}{1 + \Gamma})}^{u} y \left((1 - \Gamma) \mathbb{P}(a \mid x) + \Gamma \right) \mathbb{P}(y \mid x, a) dy \xrightarrow[\Gamma \to \infty]{} \mathbb{P}(a \mid x) \mathbb{E}[Y \mid x, a] + (1 - \mathbb{P}(a \mid x)) u \end{equation} This bound is also known as the "no assumptions bound" or "Manski bound", originally derived in [1]. Of note, with our theory, we can derive similar bounds for distributional effects. Hence, **our paper even makes non-trivial contributions to CPA**. We will add this to our paper. 2) For continuous treatments, using the CMSM we obtain \begin{equation} Q^+ = \int_{\ell}^{F^{-1}(\frac{\Gamma}{1 + \Gamma})} \frac{y}{\Gamma} \mathbb{P}(y \mid x, a) dy + \int_{F^{-1}(\frac{\Gamma}{1 + \Gamma})}^{u} y \Gamma \mathbb{P}(y \mid x, a) dy \xrightarrow[\Gamma \to \infty]{} u \end{equation} That is, for continuous treatments the corresponding "no assumptions bound" is exactly the right support point of the observed distribution (see also [2]). Hence, informative CPA for continuous treatments and continuous outcomes is **not possible without imposing stronger assumptions** (such as IVs). **Action**: We will add the derivations above to our discussion on CSA vs CPA.

Area Chair vLL72023-08-18

Acknowledging author rebuttals

Dear all, I want to thank the authors for their rebuttals and want to acknowledge that these will be taken into account. Unfortunately, 2 reviewers have still not replied to the author rebuttal and I explicitly urge them (again) to please reply to the rebuttals as soon as possible since the author/reviewer discussion period ends soon. There are still substantial discrepancies in the scores this submission has received so far and the reviewer-author discussion is crucial to clarify the different viewpoints. Best regards

Reviewer GZoa2023-08-18

Thank you for your detailed reply. Your explanations helped me to better understand the framing of sensitivity models using the weighting function. I now have a greater appreciation for the generality of this approach and am correspondingly raising my score to a 6. The reason I am not raising my score further is that I believe the experimental section could have been a bit more fleshed out. The additional IHDP results are interesting, but they appear to only compare distributional/quantile bounds against expectations for a hypothetical downstream task. Since the method proposed in this work generates sharp bounds for a variety of conditions, I would have imagined that the sharpness could be demonstrated in practice to give better bounds than previous approaches, for e.g. partially identifying conditional expectations, in semi-synthetic settings with hidden confounding,

Authorsrebuttal2023-08-19

Thank you for your response and for raising your score

Many thanks for acknowledging our rebuttal and for raising your score. Please allow us to elaborate on our experimental results. We would like to reiterate that the aim of our paper is not to improve on previous results for binary CATE but rather to generalize existing sharp bounds to other sensitivity models, causal estimands, and causal inference settings. For this purpose, we propose an entirely new approach to deriving sharp bounds in Pearl's SCM framework (see Fig. 2). We agree that generally, benchmarking with previous bounds on (semi-)synthetic datasets is desirable to evaluate performance improvement. However, in most settings where our paper has novel contributions (e.g., mediation analysis, distributional effects) **there exist currently no baselines**. **For binary CATE, we obtain exactly the same (sharp) bounds** as Dorn and Guo (2022). That is, the mathematical formulas for the bounds coincide when setting $\mathcal{D} = \mathbb{E}$ for the MSM in our Corollary 1. We believe that the fact that we obtain the same sharp bounds as previous literature is rather encouraging, and indicates (aside from our proofs and experimental results) that our approach for deriving the bounds (Fig. 2) is indeed correct. For continuous CATE, we provide a comparison in Table 1. Hence, there is no point in benchmarking our CATE bounds with other approaches for the IHDP data (binary treatment, no mediators). We thus decided to use the IHDP data to illustrate how our bounds for distributional effects can aid decision-making under unobserved confounding.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC