Optimal Treatment Allocation for Efficient Policy Evaluation in Sequential Decision Making

A/B testing is critical for modern technological companies to evaluate the effectiveness of newly developed products against standard baselines. This paper studies optimal designs that aim to maximize the amount of information obtained from online experiments to estimate treatment effects accurately. We propose three optimal allocation strategies in a dynamic setting where treatments are sequentially assigned over time. These strategies are designed to minimize the variance of the treatment effect estimator when data follow a non-Markov decision process or a (time-varying) Markov decision process. We further develop estimation procedures based on existing off-policy evaluation (OPE) methods and conduct extensive experiments in various environments to demonstrate the effectiveness of the proposed methodologies. In theory, we prove the optimality of the proposed treatment allocation design and establish upper bounds for the mean squared errors of the resulting treatment effect estimators.

Paper

Similar papers

Peer review

Reviewer bgAF6/10 · confidence 1/52023-06-27

Summary

The paper designs methods for taking actions in multiple sequential settings (Markovian, time-varying Markovian, non-Markovian) to generate high-quality data for the estimation of average treatment effect.

Strengths

1. Focuses on improving ATE estimation by improving the data collection process, as opposed to improving estimators etc, which is a nice take on this problem.

Weaknesses

None identified.

Questions

1. Theorem 1 and the implementation seems like an "explore then commit" type of a strategy seen in bandits. Is there any connection/parallels? 2. No need for the number of days in Sec 2. 3. It is not clear why running the two fixed policies sequentially one after the other cannot estimate ATE well. 4. Maybe this is easy to see, but Theorem 1's pi* is not taking one of the actions at all (e.g., if A1 happens to be 1, then A2=A1... etc). If so, why would ATE be estimated well? 5. If you are already able to estimate V_1^a in step 3 of Algo 1, can't you estimate ATE well at this point itself? 6. Clarification question: From Theorem 1, it seems like we are picking one of the actions to run for T periods (or "after burn-in" in Alg 1) that has a higher estimated variance in the rewards it obtains. This seems like a natural thing to do to estimate ATE better. Please correct me if I am wrong here.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

1: Your assessment is an educated guess. The submission is not in your area or the submission was difficult to understand. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

1. Hard to appreciate the optimal treatment strategy introduced in Sec 3, likely because the treatment is a bit terse and skips some useful discussion. 2. The data collection strategy is agnostic to the estimator used later. Is there any further improvement to be had? It would be good to see comments on this, as the estimators are already known in the literature.

Reviewer NJNM6/10 · confidence 5/52023-07-04

Summary

This paper studies optimal designs for allocating treatments so that treatment effects can be accurately estimated in online experiments. The focus of the paper is designing behavior policies for data generation to minimize the MSE of the ATE estimator when we know that dynamics may evolve through an NMDP, TMDP, or MDP.

Strengths

1. The paper studies the challenging and important problem of SUTVA violations in online experiments. Without accounting for these violations, the temporal carryover effects that occur due to sequential allocation of treatment in online experiments can yield biased treatment effect estimates. 2. The authors consider an ambitious problem of designing optimal behavior policies when data generation evolves via NMDP, TMDP, or MDP. 3. Although there are many works that focus on off-policy evaluation and estimation of treatment effects, not many of these focus on how to design the behavior policy to improve the efficiency of the policy value estimator (it is more common to assume that a fixed dataset, generated according to some prespecified or unknown behavior policy, already exists). 4. The paper has a thorough and comprehensive related work section. 5. The paper is clear and well-written.

Weaknesses

1. In Section 4, when analyzing the optimal designs for the TMDP and MDP models, the choice to restrict the treatment assignment policies that are considered to the class $\Pi^{b}$, the class of policies that randomly assign the first action and sticks with the same action for the rest of the trajectory, seems quite restrictive. In some sense, this removes some of the “dynamic” aspects of the original problem and greatly reduces the complexity of the problem. 2. It seems like the burn-in period is quite essential to estimating the ATE with the proposed designs because proposed behavior policies randomly selects an action $A_{1}$ in the first time step and then sticks with it for the remaining time steps. As a result, if we didn’t have the burn-in period, we wouldn’t be able to estimate both $V_{1}^{0}$ and $V_{1}^{1}$. The importance of the burn-in period isn’t emphasized in the text and is worth commenting on (or at least citing references that burn-in periods are necessary for consistent estimation).

Questions

1. Can the authors discuss the choice to restrict the treatment assignment policies to class $\Pi^{b}$? 2. Is it possible to obtain good estimates of the ATE without requiring any burn-in period? (If not, it would be helpful to add a short comment on this to the paper.) 3. Is a burn-in period used in the experiments? If so, how long was the burn-in period? (Can this be added to the main text?)

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

4 excellent

Presentation

4 excellent

Contribution

2 fair

Limitations

Yes.

Reviewer WpTu7/10 · confidence 4/52023-07-04

Summary

The paper studies optimal treatment allocation aiming to maximize the obtained information from online experiments to estimate treatment effects accurately. The authors propose optimal allocation strategies in a dynamic setting. These strategies are designed to minimize the variance of the treatment effect estimator when data follow a NMDP or TMDP.

Strengths

The problem that the paper investigates is important, in my opinion. The paper provides an insightful paradigm that how reinforcement learning techniques can be effectively applied in experimental design/causal inference. The authors also did a great job on giving credits to RL literature.

Weaknesses

1. There are several very relevant papers in experimental design literature that needs to be discussed carefully, for example, Bojinov et al. (2022) and Farias et al. (2022). I think some insights and the designs in this paper are closely related to these two papers. 2. From experimental design/causal inference literature, we may not only want to know an estimator of ATE, but also the corresponding confidence intervals for the estimators. I am wondering that whether the authors could have some comments on how to construct confidence intervals. 3. The definitions and relationships between $n$ and $T$ are a little bit confusing, especially when reading Section 2. I think some more descriptions on the relationship them will be helpful. From my understanding, the data structure looks very similar to “panel data” in the literature. Reference: Bojinov, I., Simchi-Levi, D., & Zhao, J. (2022). Design and analysis of switchback experiments. Management Science. Farias, V., Li, A., Peng, T., & Zheng, A. (2022). Markovian interference in experiments. Advances in Neural Information Processing Systems, 35, 535-549.

Questions

I have another minor question on the word “dynamic” in the title. Since the optimal allocation rule is $A_1=A_2=\cdots=A_T$, I am not pretty sure whether it is still suitable to call it a "dynamic" allocation.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

4 excellent

Contribution

3 good

Limitations

See previous comments.

Reviewer tKDd6/10 · confidence 2/52023-07-12

Summary

This paper considers A/B tests which are often employed for evaluating new products/treatments/policies against existing baselines. The goal is to study the optimal design of such A/B tests to maximize the information obtained and estimate treatment effect more accurately. The paper considers three possible settings governing how the data is generated: whether it follows an MDP or if it follows a time-varying MDP or a non-Markov process and presents an optimal allocation strategy for each. The paper tests these policies on a real-world sourced dataset pertaining to a ride-sharing company and shows that these approaches achieve better accuracy. Finally, the paper also establishes nice theoretical properties of these methods and derive an upper boudn for the MSE of the proposed estimator.

Strengths

– The paper studies an interesting problem that can be very useful in improving the efficiency of A/B tests that are commonly employed everywhere. – Paper is well-written. It also sufficiently talks about existing related work and distinguishes itself. – The key results in the paper are strongly grounded in theory and are supported by good results in the empirical section.

Weaknesses

– Could there be some imbalance in allocation of treatment leading to fairness issues? For example, to optimize sample efficiency, could the algorithm withhold treatment unnecessarily if the placebo group has high variance? That way it could put most of the people in placebo and almost none in treatment? Or similarly, if there is a negative treatment but with high variance, it could opt to allocate that treatment more often? It is unclear if the algorithm guards against these issues.

Questions

– Please see concerns about potential weakness mentioned in above section. Additional question below: – While the results look good in Fig 4, etc. could there be cases where errors on some of the points in proposed algorithms are really large, even though for most points they are lower than the baselines? For example, are there cases where these estimates become unstable leading to exploding errors?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

No negative social impact (other than pointed out in questions section)

Authorsrebuttal2023-08-10

Thank you and Formulas Clarification

Dear Reviewer, We extend our heartfelt appreciation for your dedicated time and valuable insights. We are writing to address a technical issue we encountered during the submission process, which led to some of the formulas in our initial rebuttal not being displayed in the final version. We deeply regret any inconvenience this may have caused. In the following, we present a revised version of the rebuttal, specifically addressing the "Confidence Intervals of the ATE Estimator," for enhanced clarity. - **Confidence Intervals of the ATE Estimator.** - This is an excellent comment! We totally agree that establishing confidence intervals for the ATE estimators holds significant importance. As elaborated further below, the derivation of a confidence interval for the proposed estimator is quite straightforward. Should our paper be accepted, we will make sure to include these discussions. - First, let us consider the settings under NMDPs in Section 3. Recall that after generating data through the proposed experimental design, we utilize the online doubly robust estimator $\widehat{\textrm{ATE}}_1$ to estimate the ATE. A key observation is that, under certain regularity conditions, the proposed estimator is asymptotically normal. More specifically, we have $$\sqrt{n - 2m_0} (\widehat{\textrm{ATE}}_1 - \textrm{ATE}) \overset{d}{\rightarrow} N(0, \textrm{EB}_1(\pi^{b*})).$$ This motivates us to consider the following Wald-type confidence interval $$ [\widehat{\textrm{ATE}}_1- \Phi^{-1} (1-\alpha/2) \sqrt{ \textrm{EB}_1(\pi^{b*})/(n - 2 m_0) }, \widehat{\textrm{ATE}}_1+ \Phi^{-1} (1-\alpha/2) \sqrt{ \textrm{EB}_1(\pi^{b*})/(n - 2 m_0) }], $$ where $\Phi^{-1}$ is the inverse cumulative distribution function of a standard normal random variable. It then suffices to estimate the asymptotic variance $\textrm{EB}\_1(\pi^{b*})$ to construct asymptotically valid confidence intervals. Notice that $\widehat{\textrm{ATE}}\_1$ can be represented as an average of martingale differences $\widehat{\textrm{ATE}}\_1 = \sum_{i=2m_0 +1}^n \psi^1_i /(n - 2m_0)$ where $$ \psi\_i^1=\sum\_{a=0}^1\frac{(-1)^{a+1}}{T} \Big[ \widehat{V}\_{1,i-1}^a(O\_1^{(i)})+ \frac{\mathbb{I}(A\_1^{(i)}=a)}{\widehat{\pi}^{b*}\_{1,i-1}(a|O\_1^{(i)})}[\sum\_t R\_t^{(i)}-\widehat{V}\_{1,i-1}^a(O\_1^{(i)})]\Big]. $$ We propose using the sample variance of $\\{\psi^1\_i\\}_i$ to estimate $\textrm{EB}_1(\pi^{b*})$. Similar to Theorem 15 of Kallus and Uehara (2022) (https://dl.acm.org/doi/abs/10.5555/3455716.3455883), we can establish the consistency of the resulting sampling variance estimator. - For TMDPs, we can similarly establish the asymptotic normality of $\widehat{\textrm{ATE}}_2$, i.e., $\sqrt{n - 2 m_0} ( \widehat{\textrm{ATE}}_2 - \textrm{ATE} ) \overset{d}{\rightarrow} N(0, \textrm{EB}_2(\pi^{b*}) ) $. The corresponding $1 - \alpha $ confidence interval can be constructed by $$ [\widehat{\textrm{ATE}}_2- \Phi^{-1} (1-\alpha/2) \sqrt{ \textrm{EB}_2(\pi^{b*})/(n - 2 m_0) }, \widehat{\textrm{ATE}}_2+ \Phi^{-1} (1-\alpha/2) \sqrt{ \textrm{EB}_2(\pi^{b*})/(n - 2 m_0) }], $$ where the unknown asymptotic variance $\textrm{EB}_2(\pi^{b*})$ can be similarly estimated via the sampling variance estimator. We are once again immensely grateful for your dedicated attention and constructive feedback. In addition, if you have any additional questions or concerns, we would be glad to hear from you during the discussion period and provide clarification.

Reviewer WpTu2023-08-11

Thank you very much for your extra efforts. The construction of the confidence interval, from my perspective, enhances the results of the paper a lot.

Reviewer NJNM2023-08-13

Thank you for these clarifications! I hope the authors include some of these discussions in their final draft, and I recommend accepting this paper.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC