> "regarding "general DG problem": thanks for pointing this out and I should not use "general" here. To be accurate, I mean these problems or datatsets that appear in many deep learning papers, e.g., ColoredMNIST, VLCS, etc. (or check the paper: In Search of Lost Domain Generalization, ICLR 2021). How does the result of this paper improve emprical performance of these tasks?"
Thank you for the clarification. We can give a concrete answer for the Colored MNIST dataset, where CRO entails a classifier with optimal worst-case generalization performance subject to variation in the correlation between the image color and label. The global rebuttal provides a description of that experiment (first bullet point and attached PDF), with figures describing the learning process of CRO as well as explicit comparisons with baseline algorithms.
More generally, recall that CRO can be viewed as a meta-algorithm that is designed to exploit background information on the discrepancies and invariances between source and target domains, and guarantees optimal classifiers subject to these constraints. In this sense, if, for a particular DG task, background information can be specified (in the form of selection diagrams), CRO can be applied and will deliver a classifier that achieves best worst-case performance, as in the Colored MNIST experiment. Note, however, that background information might not necessarily be available for an arbitrary task defined in "In Search of Lost Domain Generalization, ICLR 2021". Out-of-the-box comparisons with more data-driven baselines, such as ERM and IRM, should be considered with care as these algorithms exploit different sets of assumptions that may not be appropriate in all settings.
> "regarding qmax in Def. 4: if there is no further constraint, then there are many functions (e.g., every bounded function for some loss functions) that are partially transportable. Does this matter here?"
Our objective is to infer the tightest upper bound qmax that satisfies the constraints (observational equivalence and structural invariance) following the definition of partial transportability. This notion is useful as it characterizes statements such as "the query is partially transportable with upper bound qmax" even though bounded functions are, by definition, already bounded from above. Considering the bounds in the context of worst-case performance of classifiers for the DG task, what you mention is correct since, with symmetric 0-1 loss, a trivial upper-bound for the risk of all classifiers is $R_{P^*}(h) \leq 1$, however, this bound is not informative or useful for classification in the DG problem. The bounds achieved by Neural-TR procedure are tightest valid bounds considering the source data and the domain knowledge, and provide a basis for comparing the DG performance of candidate classifiers.