Synthetic Combinations: A Causal Inference Framework for Combinatorial Interventions

Consider a setting where there are $N$ heterogeneous units and $p$ interventions. Our goal is to learn unit-specific potential outcomes for any combination of these $p$ interventions, i.e., $N \times 2^p$ causal parameters. Choosing a combination of interventions is a problem that naturally arises in a variety of applications such as factorial design experiments, recommendation engines, combination therapies in medicine, conjoint analysis, etc. Running $N \times 2^p$ experiments to estimate the various parameters is likely expensive and/or infeasible as $N$ and $p$ grow. Further, with observational data there is likely confounding, i.e., whether or not a unit is seen under a combination is correlated with its potential outcome under that combination. To address these challenges, we propose a novel latent factor model that imposes structure across units (i.e., the matrix of potential outcomes is approximately rank $r$), and combinations of interventions (i.e., the coefficients in the Fourier expansion of the potential outcomes is approximately $s$ sparse). We establish identification for all $N \times 2^p$ parameters despite unobserved confounding. We propose an estimation procedure, Synthetic Combinations, and establish it is finite-sample consistent and asymptotically normal under precise conditions on the observation pattern. Our results imply consistent estimation given $\text{poly}(r) \times \left( N + s^2p\right)$ observations, while previous methods have sample complexity scaling as $\min(N \times s^2p, \ \ \text{poly(r)} \times (N + 2^p))$. We use Synthetic Combinations to propose a data-efficient experimental design. Empirically, Synthetic Combinations outperforms competing approaches on a real-world dataset on movie recommendations. Lastly, we extend our analysis to do causal inference where the intervention is a permutation over $p$ items (e.g., rankings).

Paper

Similar papers

Peer review

Reviewer z6nm7/10 · confidence 4/52023-06-22

Summary

The manuscript introduces a synthetic combinations estimator, which computes causal effects for unseen treatment combinations under some reasonable assumptions on the data generating process. Building on theory from potential outcomes and synthetic interventions, the authors show how to exploit information sharing across units and treatments to identify causal effects in this challenging setting. The proposed two-step algorithm is more efficient and flexible than existing alternatives.

Strengths

The topic is timely and interesting. The manuscript is exceptionally clear and well-written, which is greatly appreciated when dealing with dense formalisms. The theoretical results are strong and convincing, rooted in established results while simultaneously going beyond the current state of the art. The analysis is sound and well-motivated.

Weaknesses

My main critique of this manuscript is one I am sure the authors will have anticipated – there are no empirical results in the main text! I am aware that 9 pages is tight but the authors could have been more judicious in their selection of what to send to the appendix. I also found the material on CART to be super interesting; a shame to banish this material to the wilderness of Appendices J and K. Fortunately, the final manuscript affords one extra page. I strongly encourage the authors to move Fig. 2 to the main text and expand on this empirical evaluation. Would also be great to shoehorn in some of the CART results, but that may prove difficult. If it is impossible to squeeze everything within the limit, then the authors may want to consider repurposing this manuscript for a journal submission. There is more than enough material here for a very solid journal contribution.

Questions

See above.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

4 excellent

Contribution

4 excellent

Limitations

N/A

Reviewer rsAG7/10 · confidence 3/52023-07-03

Summary

The paper studies the problem of estimating potential outcomes in the presence of a combinatorial number of intervention choices. Under some assumptions, they propose a two-phased algorithm "Synthetic Combinations": first exploit structure across combinations of interventions (via "horizontal regression") and then exploit structure across units (via "vertical regression"). Experiments are given in the appendix.

Strengths

The proposed algorithm is clean and intuitive. It also seems to scale nicely with the number of intervention combinations in experiments. While the theoretical guarantees rely heavily on a bunch of assumptions, Section 7 proposes an experimental design framework which ensures that an important set of assumptions (existence of donor units) will be met with high probability. In fact, I strongly propose that the authors rephrase their paper to highlight this; otherwise it is hard to believe that their work will useable as it is highly unlikely that all the required assumptions are met in practice without having control in assigning interventions to the units.

Weaknesses

I did not check all the proofs in detail, but I do not see any glaring weaknesses. There are a lot of assumptions and it is highly unlikely that all the required assumptions are met in practice (Section 7 helps to mitigate some of these concerns). I am skeptical about the low-rank assumption on the matrix of Fourier coefficients $A$. While it is true low-rank assumptions are common in prior matrix completion settings, they usually directly consider the matrix at hand and not transform it into the Fourier space first. For example, Lines 655-657 in the appendix writes "This missingness pattern where outcomes with larger absolute values are observed is common in applications such as recommendation engines, where we are only likely to observe ratings for combinations that users either strongly like or dislike". The corresponding missingness pattern in the problem studied here is on the $N$-by-$2^p$ matrix. It is unclear to me why it should be believable that the transformed space is low-rank. The authors ought to justify this, ideally with practical examples/settings, or risk diminishing the impact of their contributions.

Questions

Line 83: By "equivalent", do you mean that they proved equivalence between the two problems via reductions, or do you mean "equivalent" in a colloquial sense of the word? Assumption 3.1: As discussed in the weaknesses, it is unclear to me why this model is interesting or justified. Of course, this work can be appreciated under the restriction of this assumption, but it will greatly weaken the contributions. I am more than happy to increase my "contribution" score if the authors provide sufficient justification for the low-rank assumption. Type on Line 183: double "exists" Motivating example on Line 190: I don't understand why this motivates the existence of donor units when the paper has thus far repeatedly claim to allow unobserved confounding. If we allow interventions to be arbitrarily assigned to units, it is unclear why we should believe that donor units exist. The "correct" way to justify should be to say that there is an experimental design that ensures the existence of donor units, and then refer to Section 7. Determining donor set on Line 246: This feels very ad-hoc. As it is unlikely that donor units will exist if we allow arbitrary experiments, I feel that this paragraph could be removed once the authors reorder their paper to place more emphasis on the experimental design proposed in Section 7. Subsection on Additional Assumptions: I feel that "so-and-so also has such an assumption" is not sufficient discussion of assumptions. Firstly, "so-and-so" may have the assumptions under different contexts (e.g. see my complaint about low-rank assumption in the Weaknesses section) so it is unclear why such assumption is justified in the setting studied in this paper. Secondly, the discussion should explain "what goes wrong" if one particular assumption is violated, or why we should expect any particular assumption to hold in practice. As mentioned several times by now, one "partial fix" is to emphasize that experimental design of Section 7 guarantees some assumptions with high probability. That is, "Synthetic Control" should be used in conjunction with the experimental design proposed in Section 7.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

4 excellent

Limitations

Nil.

Reviewer qb2F7/10 · confidence 3/52023-07-07

Summary

The goal paper studies the problem of recovering $N\times 2^p$ unit-specific outcomes for N heterogeneous units and any combination of p possible interventions from small number of experiments and observations. Prior to this work, the problem has been studied under assumptions of latent similarity or regularity in how combinations of interventions interact, as well as some other setups. In the setup when one assumes latent similarity, namely that the matrix of Fourier coefficients across units has rank <= r, the problem is reduced to matrix completion, and hence all causal outcomes can be recovered from $O(poly(r)\times (N+2^p))$ observations. In the case, when regularity in intervention interactions is assumed it is known that $O(Ns^2p)$ measurement are sufficient, where s is the sparcity parameter of coefficients in the Fourier expansion of the potential outcomes. This paper studies the problem in the case when both latent similarity and intervention regularity is assumed. Under both assumptions the paper proposes an algorithm that recovers all $N\times 2^p$ causal outcomes from $O(N\times (s^2 + p))$ measurements.

Strengths

The problem of estimating causal outcomes under a combination of interventions is a notoriously hard problem with various applications. One complication is that it is usually expensive\impossible to run many experiments to measure the effects caused by interventions. Hence, understanding how to set up experiment that will require the minimum amount of measurements is of great importance. This paper proposes an algorithm, called Synthetic Combinatorics, that provably recovers all $N\times 2^p$ unit-specific outcomes under $2^p$ interventions under a combination of two widely accepted assumptions from $O(N\times (s^2 + p))$, which is a significant improvement over the prior work. The paper also provides statistical estimates for the number of samples needed for every experiment to achieve desired accuracy of the recovery. The paper is well-written and as far as I can judge is correct, though I did not read the proofs carefully.

Weaknesses

It is not completely clear how realistic is the scenario when latent similarity and intervention regularity holds simultaneously. I can imagine that in some datasets one or the other may hold, while both assumptions at the same time may not hold. The paper will benefit significantly from experiments on real-world datasets, that can confirm that theoretical assumptions are realistic and we indeed see improvement in the number of measurements needed.

Questions

Can you provide some intuition why you believe that the assumptions needed for Synthetic Cobminatorics to work are expect to hold for real-world datasets?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

-

Reviewer Wabz7/10 · confidence 1/52023-07-27

Summary

__Disclaimer__: This is my first time reading & reviewing a paper from the field of Combinatorial Interventions. My expertise is in NLP. This work studies latent structure across units and combinations of interventions, assuming similar outcomes across units and regular interaction. An estimation procedure, Synthetic Combinations, is proposed, establishing finite-sample consistency under precise conditions. This work also uses methods to reduce errors in variables and provides a possibility of model-agnostic analysis.

Strengths

* All the proofs and other mathematical explanations are clear, but I'm not able to understand them properly because of no expertise in that field.

Weaknesses

*

Questions

NA

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

1: Your assessment is an educated guess. The submission is not in your area or the submission was difficult to understand. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

4 excellent

Contribution

4 excellent

Limitations

NA

Reviewer rsAG2023-08-10

Thank you for your patience and effort to clear my doubts and misunderstandings. Also, I appreciate the effort that the authors took to perform additional experiments --- it must have been tough to do so in such a short period of time! I am very satisfied with the detailed response and have updated the scores accordingly :) Please kindly incorporate some of the discussions here into your revision. Thanks!

Authorsrebuttal2023-08-14

Thank you for taking the time to comment in such detail on our paper, and for reading our response. Your feedback was very helpful in improving our paper, and we will include them in our revision. Thank you for increasing your score!

Authorsrebuttal2023-08-14

We thank the reviewer again for their thoughtful comments. We hope that they have had a chance to review our response to their specific concerns, and our real-world experiments in the global response where we demonstrate the efficacy of Synthetic Combinations over baselines methods, and that our key modeling assumptions (i.e., low-rank and sparsity) hold. Please let us know if there is anything else we can do to address your concerns, and we hope you improve your score.

Authorsrebuttal2023-08-14

We thank the reviewer again for their thoughtful comments. We hope that they have had a chance to review our response to their specific concerns, and our real-world experiments in the global response where we demonstrate the efficacy of Synthetic Combinations over baselines methods, and that our key modeling assumptions (i.e., low-rank and sparsity) hold. Please let us know if there is anything else we can do to address your concerns, and we hope you improve your score.

Authorsrebuttal2023-08-14

We thank the reviewer again for their thoughtful comments. We hope that they have had a chance to review our response to their specific concerns, and our real-world experiments in the global response where we demonstrate the efficacy of Synthetic Combinations over baselines methods, and that our key modeling assumptions (i.e., low-rank and sparsity) hold. Please let us know if there is anything else we can do to address your concerns, and we hope you improve your score.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC