continuation of response to contributions question
> pseudo labels as well as the meta-learning part are existing techniques
**(3)** Successfully translating ideas from one ML subfield to another, and discovering that they unlock entirely new capabilities, is non-trivial and a contribution in its own right. For example, the Transformer architecture was originally introduced in 2017 for NLP. It wasn't until 2020 that its impact for computer vision was recognized with the landmark Vision Transformer (ViT) paper [7]. Similarly, progress in the subfield of Graph Neural Networks was sparked by the discovery that word embeddings (word2vec) can be extended to embed nodes in a graph (node2vec) [8].
In the same spirit, developing a framework that allows for pseudo-outcome labels and meta-learning to be (successfully) applied to zero-shot CATE estimation requires careful design considerations. *This is because pseudo-outcome labels and meta-learning cannot be applied to zero-shot CATE estimation out-of-the-box.* Our key insight in designing the CaML framework is that, rather than training on the raw (W, X, Y) triplets, we can synthesize natural experiments from the data to construct a distinct CATE estimation task for each training intervention. This allows us to (a) unlock zero-shot capability for many recently developed pseudo-outcome estimators which were previously restricted to studying single interventions in isolation (b) leverage meta-learning algorithms (i.e. Reptile) for meta-model training, which requires task-structured data. We discover in our experiments that both of these factors contribute to CaML’s strong empirical performance and allows it to outperform all baselines and existing methods.
**(4)** Finally, our theoretical contribution is to bound the zero-shot CATE prediction error as a function of several key parameters (e.g. how informative intervention information $w$ is of the true treatment effects). Compared to a standard generalization bound which usually has a $\sqrt{1/n}$ term, our main technical innovation involves bounding the variance by the smoothness of the function class plus Poincaré-type inequalities. When $\beta$ in Theorem 1 is much smaller than $1$ we achieve a tighter bound. This is unique to the causal inference setting in which we are learning a function $f = \Psi(w,x) \to \tau$ among a family $\mathcal{F}$, where $w$ is intervention information (e.g. a drug’s attributes) and $x$ is individual features (e.g. patient medical history), and $f$ is assumed to be smooth with respect to $w$.
**References**
[1] Ruddigkeit, Lars, et al. "Enumeration of 166 billion organic small molecules in the chemical universe database GDB-17." Journal of chemical information and modeling 52.11 (2012): 2864-2875.
[2] Curth, Alicia, and Mihaela van der Schaar. "Nonparametric estimation of heterogeneous treatment effects: From theory to learning algorithms." AISTATS. (2021).
[3] Nie, Xinkun, and Stefan Wager. "Quasi-oracle estimation of heterogeneous treatment effects." Biometrika (2021).
[4] Kennedy, Edward H. "Towards optimal doubly robust estimation of heterogeneous causal effects." arXiv (2020).
[5] Curth, Alicia, et al. "Really doing great at estimating CATE? a critical look at ML benchmarking practices in treatment effect estimation." NeurIPS. (2021).
[6] Yadlowsky, Steve, et al. "Evaluating treatment prioritization rules via rank-weighted average treatment effects." arXiv preprint arXiv:2111.07966 (2021).
[7] Dosovitskiy, Alexey, et al. "An image is worth 16x16 words: Transformers for image recognition at scale." ICLR (2021).
[8] Grover, Aditya, and Jure Leskovec. "node2vec: Scalable feature learning for networks." KDD. (2016).