Partial Transportability for Domain Generalization

A fundamental task in AI is providing performance guarantees for predictions made in unseen domains. In practice, there can be substantial uncertainty about the distribution of new data, and corresponding variability in the performance of existing predictors. Building on the theory of partial identification and transportability, this paper introduces new results for bounding the value of a functional of the target distribution, such as the generalization error of a classifier, given data from source domains and assumptions about the data generating mechanisms, encoded in causal diagrams. Our contribution is to provide the first general estimation technique for transportability problems, adapting existing parameterization schemes such Neural Causal Models to encode the structural constraints necessary for cross-population inference. We demonstrate the expressiveness and consistency of this procedure and further propose a gradient-based optimization scheme for making scalable inferences in practice. Our results are corroborated with experiments.

Paper

Similar papers

Peer review

Reviewer SwAx6/10 · confidence 3/52024-07-09

Summary

This paper tackles to problem of transportability in domain generalization. Multiple datasets are provided together with graphical information stipulating which causal mechanisms are shared across domains and which mechanisms are going to remain invariant in the target domain. The goal is then to estimate the error committed by a given classifier in the target domain. More specifically, the authors propose a way to estimate the worst-case error of the classifier given the observed distributions as well as the graphical information. A first result shows how the problem can be simplified using canonical SCMs and a second result shows how Neural Causal Models (NCM) can be used to estimate this bound. Furthermore, an approach named Causal Robust Optimization (CRO) is proposed to learn a classifier that minimizes the worst-case risk. Experiments experiments with the NCM approach to estimate the upper-bound of various classifiers, but does not tackle the CRO approach to learn a robust optimizer.

Strengths

- The paper reads nicely (even for a fairly technical paper) and gives a good background on SCMs and graph terminology for the non-expert. The choice of notation is good and remains consistent throughout the paper. - The problem appears to be very important, and the approach to tackle it seems principled. - The approach appears to be novel and interesting. However, I don't have a good knowledge of the literature on transportability and domain generalization and thus my judgment's credibility is limited. - Many examples are provided to make the discussion more concrete, I enjoyed that.

Weaknesses

- Some theoretical results, in their current form, seems unlikely to be true. Specifically in Theorem 2, I was surprised to see that this result does not mention anything about the expressive power of the NCM, what if the neural networks do not have enough capacity to express the causal mechanisms? Clearly what you would end up with wouldn’t be an upper bound right? I was also surprised to see that the optimization problem is formulated in a finite-sample fashion, but the conclusion concerns the actual expectation (i.e. not finite-sample), this sounds unlikely, no? Don’t you need to take a limit as the dataset grows for this bound to hold? - Confusion surrounding canonical SCM: At line 181, it is mentioned that [41] showed that every SCM M can be written in a canonical form (with discrete exogenous variables)... However, after quickly skimming through the paper, I could not find something resembling that statement. What am I missing? Overall I thought the discussion on canonical SCM was confusing. I’m left unsure as to which type of SCM can be cast to a canonical form (definition 5) and which paper showed this fact. - Some section were hard to parse. Mainly lines 270 to 289, including Algorithm 1. I'm not sure I got the point. I was following up to that point. I give some minor suggestions to improve clarity below. - Algorithm 2 (Causal Robust Optimization) is interesting, but it looks quite costly. Do the authors believe it could be made applicable? I was a bit disappointed to see it was not implemented in the experimental section. It would be nice to have experiments for CRO. Could this approach be applied to the Coloured MNIST dataset? - The background is very well written, but the end of the paper felt a bit less polished. Since these are the most important part, I believe they deserve more space. For instance, I had a hard time understanding the experiments with colored MNIST. Minor: - Example 1: Why using arrows for assignment? The symbol “:=” was used earlier… - Definition 3: Should S_{ij} be in fact S_i here? (line 125) - Example 2: Long sequences of “if”s would benefit from some punctuation. The “circled plus sign” is an XOR right? - Definition 4: Might be helpful to spell out \mathbb{M}_0 completely in (4), since \mathcal{M}^*_0 appears in the expectation but is not defined (the reader has to deduce that this is the target SCM belonging to \mathbb{M}_0) - Definition 5: might be useful to end with “where h_V^{(r_V)} is a function from supp_{pa_V} to supp_V”. I’m actually not certain the very last sentence adds something to the definition, since it is already obvious that f_V(pa_V, r_V) is a function of pa_V for all r_V… I’m also a bit confused by this sentence: “The set of endogenous variables V is discrete.” What does it mean for a set to be discrete? Do you mean the endogenous variables are discrete? Also, I’m a bit confused by the way the word “support” is used here. Usually it means the set of values for which the random variable has positive probability (at least for discrete variables that’s the definition), but here I believe this is not how “support” is used, correct? - Theorem 1: I believe the first constraint should not include the case i = *, no? I thought P* was not observed. In fact, you don’t have the case i=* on line 248 and in Theorem 2 - Figure 1 is unclear, what does the color mean? - Line 353: Bayesian inference procedure?

Questions

See above.

Rating

6

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

I didn't find any discussion of the limitations of the approach in the paper.

Reviewer fw9Z6/10 · confidence 1/52024-07-13

Summary

The paper extends the formulation of canonical models to encode the constraints for the transportability tasks. Then it adapts Neural Causal Models for the transportability task and introduces an iterative method Causal Robust Optimization to find a predictor with the best worst-case risk.

Strengths

- The paper presents a unified framework for addressing transportability problems using canonical models and neural causal models. These transportability problems are broad and highly relevant in the field of machine learning. - I appreciate the examples illustrated throughout the main text, as they greatly help the reader understand the definitions and intuitions. - The theorems are solid and the experiments align well with the theories.

Weaknesses

Disclaimer: I am not familiar with this field, so my observations and suggestions are based on my general understanding on the paper. - Some definitions are not provided, making it difficult for readers to understand certain parts of the paper. For example line 133, 134 $\bigoplus$. Minor: 1. Page 1 line 9, “such as”.

Questions

1. What does $\bigoplus$ mean? 2. Could you provide some experimental results for CRO and also compare it with other algorithms?

Rating

6

Confidence

1

Soundness

4

Presentation

3

Contribution

2

Limitations

The authors have adequately addressed the limitations.

Reviewer S6HF6/10 · confidence 2/52024-07-13

Summary

This paper studies the problem of domain generalization through the lens of partial transportability, and introduces some new results for bounding the value of a functional of the target distribution, given data from source domains and assumptions about the data generating mechanisms. Authors adapt existing parameterization schemes such as Neural Causal Models to encode the structural constraints necessary for cross-population inference. Some experiments and examples are also provided.

Strengths

- Introducing partial transportability and canonical SCM to domain generalization is novel. - The theoretic results are sound and have complete proofs. They are potentially useful.

Weaknesses

- The paper organization and presentation seem complicated to convey the idea of the paper (see below). - It is not clear how this bound can in turn help improve general DG problems.

Questions

Based on my understanding, authors consider bounding the queries of generalization errors in non-transportable settings, by obtaining a bound for the worst-case losses. Authors use the formulation of canonical models as a way of encoding constraints to derive such a bound. 1. I believe there are existing works that assume some prior information regarding target domain and then derive bounds for the target loss in DG problems. If my understanding is correct, then I would like to treat the current work as **an alternative way** of encoding information about the target domain. Then the current way of presentation and organization indeed make it complicated to understand the paper. In particular, I feel that examples 1 and 2 somehow distract reader's attention, while it is better to directly formulate the central problem of the paper. 2. Other questions: 1. line 125: what does $j$ indicate in the notation $S_{ij}$? 2. definition 4: is there any constraint on $q_{max}$ ? there may exist trivial but useless upper bound for some loss functions. 3. Suggest give more introductions about canonical SCM, to make the paper more self-contained. 3. about Colored MNIST experiment: I don't get how this experiments serves. Do you want to show the proposed bound is correct? However, there is no mention of the value of $R_{P*}(h)$ here.

Rating

6

Confidence

2

Soundness

3

Presentation

2

Contribution

3

Limitations

see above.

Authorsrebuttal2024-08-09

> "regarding "general DG problem": thanks for pointing this out and I should not use "general" here. To be accurate, I mean these problems or datatsets that appear in many deep learning papers, e.g., ColoredMNIST, VLCS, etc. (or check the paper: In Search of Lost Domain Generalization, ICLR 2021). How does the result of this paper improve emprical performance of these tasks?" Thank you for the clarification. We can give a concrete answer for the Colored MNIST dataset, where CRO entails a classifier with optimal worst-case generalization performance subject to variation in the correlation between the image color and label. The global rebuttal provides a description of that experiment (first bullet point and attached PDF), with figures describing the learning process of CRO as well as explicit comparisons with baseline algorithms. More generally, recall that CRO can be viewed as a meta-algorithm that is designed to exploit background information on the discrepancies and invariances between source and target domains, and guarantees optimal classifiers subject to these constraints. In this sense, if, for a particular DG task, background information can be specified (in the form of selection diagrams), CRO can be applied and will deliver a classifier that achieves best worst-case performance, as in the Colored MNIST experiment. Note, however, that background information might not necessarily be available for an arbitrary task defined in "In Search of Lost Domain Generalization, ICLR 2021". Out-of-the-box comparisons with more data-driven baselines, such as ERM and IRM, should be considered with care as these algorithms exploit different sets of assumptions that may not be appropriate in all settings. > "regarding qmax in Def. 4: if there is no further constraint, then there are many functions (e.g., every bounded function for some loss functions) that are partially transportable. Does this matter here?" Our objective is to infer the tightest upper bound qmax that satisfies the constraints (observational equivalence and structural invariance) following the definition of partial transportability. This notion is useful as it characterizes statements such as "the query is partially transportable with upper bound qmax" even though bounded functions are, by definition, already bounded from above. Considering the bounds in the context of worst-case performance of classifiers for the DG task, what you mention is correct since, with symmetric 0-1 loss, a trivial upper-bound for the risk of all classifiers is $R_{P^*}(h) \leq 1$, however, this bound is not informative or useful for classification in the DG problem. The bounds achieved by Neural-TR procedure are tightest valid bounds considering the source data and the domain knowledge, and provide a basis for comparing the DG performance of candidate classifiers.

Reviewer fw9Z2024-08-07

After Rebuttal

Thanks to the authors for their reply. I appreciate the experiments they add. I will keep my score.

Reviewer S6HF2024-08-08

Thanks for your reponse that mostly addresses my concerns. - regarding "general DG problem": thanks for pointing this out and I should not use "general" here. To be accurate, I mean these problems or datatsets that appear in many deep learning papers, e.g., ColoredMNIST, VLCS, etc. (or check the paper: In Search of Lost Domain Generalization, ICLR 2021). How does the result of this paper improve emprical performance of these tasks? - regarding $q_max$ in Def. 4: if there is no further constraint, then there are many functions (e.g., every bounded function for some loss functions) that are partially transportable. Does this matter here?

Authorsrebuttal2024-08-13

With the discussion period coming to its end, we were wondering whether you had a chance to check our rebuttal. We hope to have answered all concerns to your satisfaction. If not, please don't hesitate to get in touch if there is any concern we could still help to clarify. Thank you again for your time and attention.

Reviewer SwAx2024-08-13

Thanks for the very detailed rebuttal. This is good work so I believe this work should be accepted. I maintain my score.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC