Optimal Top-Two Method for Best Arm Identification and Fluid Analysis

Top-$2$ methods have become popular in solving the best arm identification (BAI) problem. The best arm, or the arm with the largest mean amongst finitely many, is identified through an algorithm that at any sequential step independently pulls the empirical best arm, with a fixed probability $β$, and pulls the best challenger arm otherwise. The probability of incorrect selection is guaranteed to lie below a specified $δ>0$. Information theoretic lower bounds on sample complexity are well known for BAI problem and are matched asymptotically as $δ\rightarrow 0$ by computationally demanding plug-in methods. The above top 2 algorithm for any $β\in (0,1)$ has sample complexity within a constant of the lower bound. However, determining the optimal $β$ that matches the lower bound has proven difficult. In this paper, we address this and propose an optimal top-2 type algorithm. We consider a function of allocations anchored at a threshold. If it exceeds the threshold then the algorithm samples the empirical best arm. Otherwise, it samples the challenger arm. We show that the proposed algorithm is optimal as $δ\rightarrow 0$. Our analysis relies on identifying a limiting fluid dynamics of allocations that satisfy a series of ordinary differential equations pasted together and that describe the asymptotic path followed by our algorithm. We rely on the implicit function theorem to show existence and uniqueness of these fluid ode's and to show that the proposed algorithm remains close to the ode solution.

Paper

Similar papers

Peer review

Reviewer mata5/10 · confidence 4/52024-06-27

Summary

This paper considers the top-2 algorithm for identifying the best arm in multi-armed bandits. The authors introduce a new approach for determining the optimal $\beta$ for sampling the empirical best and best challenger arms. This novel method relies on a function of allocations anchored at a threshold, which is simple in concept.

Strengths

1. The paper introduces a novel method for determining the optimal $\beta$ in Top-2 algorithms, which has been a popular problem in the field. Empirical results demonstrate that the proposed method offers improved sample complexity and computational efficiency compared to existing approaches. 2. The analysis leverages fluid dynamics and the implicit function theorem to establish the algorithm's optimality, marking a unique and intriguing contribution to the field of bandit problems.

Weaknesses

1. One weakness of the paper is the use of the forced exploration stage in the AT2 algorithm (Line 1-2), which ensures that all the arm means are estimated with sufficient accuracy. It is not clear whether this is crucial for the effectiveness of the algorithm or simply for the ease of theoretical proof. Indeed, forced exploration might harm the empirical performance of BAI but significantly simplifies the asymptotic theoretical analysis. 2. The second weakness of the paper is that all the analyses are asymptotic, although I understand that achieving non-asymptotic results might be challenging for the top-2 algorithm.

Questions

1. Line 634-637: What about a Bernoulli distribution with mean zero? 2. Line 144: Another widely used definition for a $\delta$-correct algorithm is $\mathbb{P}\left(\tau_\delta<\infty\right) = 1$ and $\mathbb{P}\left( k_{\tau_\delta} \neq 1\right) \leq \delta$. Are there any concerns with this definition? 3. It is surprising that TCB does not satisfy the asymptotic optimality conditions. Have you verified the correctness of the implementation? Additionally, did you use the same forced exploration rule and stopping rule for all algorithms in the numerical results? Finally, I suggest including a theoretical analysis of the computational complexity for the proposed algorithm.

Rating

5

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

The main contributions of this work are theoretical in nature. I don't foresee any direct negative social impact.

Reviewer Uo1U6/10 · confidence 4/52024-07-04

Summary

The paper investigates the problem of fixed-confidence BAI in stochastic multi-armed bandits. Simply stated, the problem entails finding the best arm (the arm with the largest mean reward) as quickly as possible, subject to an upper bound on the error probability. There exist a plethora fo works that investigate this problem, derive problem instance-dependent lower bounds on the asymptotic growth rate of the expected stopping time, and devise algorithms that achieve the asymptotic lower bound; here, the asymptotics is one of vanishing error probabilities. One class of efficiently implementable algorithms for this problem that has gained significant attention in the recent years is that of top-2 sampling algorithms. The basic premise on which such algorithms function is to pull the empirical best arm at any given time with a certain constant probability, say $\beta$, and to pull the empirically second-best arm (or the challenger arm) with the remaining probability. Despite its elegance and ease of implementability, the aforementioned $\beta$-top-2 sampling algorithm an algorithm falls short of achieving the instance-dependent lower bound. The key to achieving the lower bound, as it turns out, is to set $\beta=\beta^*$, where $\beta^*$ depends on the underlying problem instance and is therefore unknown to the algorithm. While several attempts to learn the value of $\beta^*$ as part of the algorithm have taken place in the recent past, the current work accomplishes the same objective elegantly, using the theory of ODEs. Notably, the paper proposes an algorithm for BAI, and analyses the fluid dynamics (ODE) corresponding to a continuous version of the same. Implicit function theorem is used to demonstrate the existence and uniqueness of the solutions to the ODE, and to further demonstrate that the arm allocations under the algorithm closely follows the trajectory coming from the ODE. It is then argued that the limiting solution of the ODE (and hence the asymptotic arm allocations under the algorithm) matches with the optimum allocations corresponding to the underlying instance, thereby proving the asymptotic optimality of the proposed algorithm.

Strengths

The use of the theory of ODEs and the implicit function theorem to demonstrate the asymptotic optimality of the proposed algorithm is the novel aspect of this paper. To the best of my knowledge, such an approach to the demonstration of asymptotic optimality does not exist in the literature on fixed-confidence BAI. Furthermore, while the concept of *anchor function* appearing in the paper has its roots in Garivier and Kaufmann's paper on BAI, a careful analysis of its behaviour using the theory of ODE to demonstrate asymptotic optimality of the proposed algorithm is impressive. (I must acknowledge, though, that I was unable to go through the extensive and detailed appendix due to paucity of time.)

Weaknesses

The writing of the paper can be slightly improved. While the authors have done a commendable job in explaining the intuition behind their detailed proofs, I have some minor suggestions that might possibly improve the readability of the paper. 1. In the paragraphs leading up to Proposition 2.1, the authors introduce the notation $x\_{1,a}=\dfrac{\omega\_1 \ x\_1 + \omega\_a \ x\_a}{\omega\_1 + \omega\_a}$. Shortly thereafter, the authors use the phrase "define $N\_{1,1}$ as the value of $N\_1$ at which $\sum\_{a \neq 1} \dfrac{d(\mu\_1, x\_{1,a})}{d(\mu\_1, x\_{1,a})}=1$ for a given allocation $\boldsymbol{N}\_{\bar{B}^c}$", without stating the (new) definition of $x\_{1,a}$. The actual definition of $x\_{1,a}$ only appears later, in the first part of Proposition 2.1. I suggest that the authors move the definition of $x\_{1,a}$ to an earlier paragraph if possible. Some minor comments: 2. "alocation" in line 160 --> "allocation". 3. $f(\boldsymbol{\mu}, a, N)$ in line 291 --> $f(\boldsymbol{\mu}, a, \boldsymbol{N})$.

Questions

Can the authors explain what role the anchor function $g$ plays at a higher level. It seems almost unintuitive that this function, together with the empirical means and allocations, should dictate the sampling of the best empirical arm or challenger arm at any given time (items 3,4 in lines 208-211), in order to match with the optimal allocations in the long run. For a contextual comparison, in the TCB and ITCB algorithms of [21], the place of the anchor function is taken by the empirical problem complexity. Given that the empirical terms are expected to be close to their true values in the long run, the empirical problem complexity is likely to be close to the true complexity. It is then intuitive to expect that any sampling rule that is based on the empirical problem complexity, is likely to lead to sampling the arms according to the optimal allocation in the long run. Along these/similar lines, can the authors provide a high-level intuition for why looking at the anchor function and its long-term behaviour is pivotally connected to learning the optimal $\beta$ and thereby matching the arm selection frequencies with the optimal allocation?

Rating

6

Confidence

4

Soundness

4

Presentation

2

Contribution

4

Limitations

NA.

Reviewer jbje5/10 · confidence 2/52024-07-07

Summary

This paper considers the problem of identifying an optimal "top-2" algorithm for the best arm identification problem in the bandits framework. The question of how to allocate pulls between incumbents and challengers is considered. An algorithm for achieving the best balance is described.

Strengths

"Top-2" approaches for best arm identification are powerful and increasingly popular, so this paper's focus is an important area of study. Posing the problem in terms of fluid dynamics could potentially be a powerful way of modeling the problem.

Weaknesses

I found the paper to be generally very hard to follow. The language is awkward in many places. I do not find that the introduction makes it clear how the paper's contributions are achieved. Explanations are very math-heavy and hard to follow and I find the theoretical results (e.g., Proposition 2.1) are not understandable. The paper also references many concepts without explaining them well (e.g., IFT, GLLR, ITCB). The presentation of this paper needs to be significantly improved before it is ready for publication.

Questions

If we were to pull both of the top-2 arms instead of randomizing between them, how different would the sample complexity be?

Rating

5

Confidence

2

Soundness

2

Presentation

1

Contribution

1

Limitations

NA

Reviewer Aqmy6/10 · confidence 3/52024-07-08

Summary

This paper focuses on best arm identification (BAI) under the fixed confidence setting. It assumes the underlying distributions are from the single parameter exponential function. It discusses the problem of how to find the optimal $\beta$ for Top-Two type of algorithm for BAI, where $\beta$ denotes the probability of pulling the empirical best arm at each step. It proposes two algorithms, called Anchored Top-2 (AT2) algorithm and an improved version called Improved AT2 (IAT2) algorithm. It shows that both of the algorithms reaches the optimal sample complexity when $\delta \to 0$. A key idea the authors propose is to identift the underlying fluid dynamics using the ODEs tracked by the proposed algorithms. This idea helps to prove the convergence of the algorithms. It also does simulations to show the index values for each arm at different time steps for AT2 and compares the sample complexity of AT2 with other popular Top-Two algorithm.

Strengths

1. The paper discusses an important question which is to find the optimal $\beta$ in the Top-Two algorithm and shows a sound solution (AT2 algorithm) to solve this problem. 2. The contributions are solid and they novelly use the fluid dynamics to describe the behaviors of the proposed algorithms. 3. The tools for proving the upper bound of the sample complexity of the proposed algorithms are also novel enough.

Weaknesses

1. The paper is rather technical and lacks some intuitive explanations of different parts. It would be nice to include a picture to desribe the actual behavior of the paths captured by the ODEs if possible. Or just provide more intuitions for the steps of the algorithms. 2. The experiments lack some more explanations to discuss the meanings behind.

Questions

1. possible typos: "There" -> "Their" in line 88? 2. What is $a$ in the equation at the end of page 3? 3. Why the threshold for function $g(\cdot)$ in the algorithm is 0? Is it possible or trivial to change to other thresholds? 4. Why the stopping condition is ignored as mention in line 249 for the fluid dynamics?

Rating

6

Confidence

3

Soundness

4

Presentation

3

Contribution

3

Limitations

See weakness.

Reviewer 4f2z7/10 · confidence 4/52024-07-17

Summary

The paper is about fixed confidence bandit best arm identification. The authors study the top-two family of algorithms. Those algorithms identify at each time step a leader and a challenger arm, and sample one of those two arms: typical top-two algorithms in the literature sample the leader with a fixed probability $\beta$ chosen in advance. A top-two algorithm that uses a fixed $\beta$ cannot have optimal expected sample complexity for all problems, since the optimal proportion of samples for the best arm might not be $\beta$. In order to derive an asymptotically optimal top-two algorithm, the authors introduce an adaptative choice of $\beta$, and study its dynamic through the study of a continuous time fluid dynamic.

Strengths

- The method proposed is the first fully adaptive choice of the proportion $\beta$ that ensures asymptotic optimality for all single parameter asymptotic families. Another optimal method was already proposed in [28], but only for Gaussian distributions. - The fluid dynamic model is particularly interesting: it gives a good intuition of why the dynamic for $\beta$ converges to the right proportions. It is also a very original approach to the study of bandit identification algorithms. - The authors propose a full algorithm (leader and challenger choice as well as the new adaptive proportions $\beta$), but the new adaptive proportions could presumably be used with other choices for the leader and challenger, and could become a generic tool for asymptotically optimal top-two algorithms. - The paper is clear and explains the idea behing the new method very well.

Weaknesses

- The analysis is very asymptotic. It uses that after a while, the forced exploration of steps 1 and 2 in the algorithm ensures that all means are well estimated. That implicitly requires that $\delta$ should be small enough that the stopping condition is not attained before that time. Most of the literature on top-two algorithms is also asymptotic so that is not very surprising, but there are also non-asymptotic results (for fixed $\beta$): see [Jourdan & Degenne, Non-asymptotic analysis of a ucb-based top two algorithm]. - the algorithm AT2 uses forced exploration, which could hurt in practice: if the parameter $\alpha$ is set too high, the algorithm will explore some bad arms too much. On the other hand if it is set too low, the algorithm can get stuck sampling sub-optimal arms. Indeed, if $\alpha$ is zero the algorithm is close EB-TC from [16] which can have horrible practical performance, and the new algorithms will likely get stuck in the same way: if there is one good arm and two bad arms with equal means but the good arm gets low empirical mean at the beginning, the algorithm will sample the two bad arms forever (see [16]). IAT2 should not suffer from the same issue, thanks to its added exploration mechanism. I did not find the value of $\alpha$ used in the experiments and the effect of that parameter was not studied experimentally. - The analysis is very involved and might be hard to reuse or extend in other contexts. In particular, it uses a lot the particular shape of the optimality conditions for the sampling proportions. It is not clear if the right properties to use the same kind of technique will hold in other settings like structured bandits. This is more an open problem than a weakness of the paper. Remarks and typos: - A remark about this review: the paper including the appendix is 75 pages long, hence I could not check every proof. - An algorithm $\beta$-EB-TCB is mentionned repeatedly, supposedly coming from [16]. There is no algorihtm called $\beta$-EB-TCB in [16]. Is it $\beta$-EB-TC? Likewise, $\beta$-EB-ITCB does not exist in that paper: is it $\beta$-EB-TCI?

Questions

- The algorithm uses forced exploration to ensure that every arm is sampled some minimal amount (steps 1 and 2, lines 206 and 207). Suppose that instead, we use some variant of top-two without forced exploration, with the adaptive $\beta$ that you propose, and prove that it naturally explores (we prove a "sufficient exploration" lemma as in [16]]). That is not possible with AT2 without forced exploration unless all arms have different means since it uses greedy leader and challenger, but it could be possible with IAT2. Would the analysis still apply? Or is the forced exploration also used for something else in the analysis? - The upper bound on the time needed to reach proportions close to the optimal proportions depends on the inverse of the minimal optimal proportion over the arms (line 262, line 321). That upper bound can be very large if the minimum is close to 0, which can happen if there is a very bad arm. Can this be avoided?

Rating

7

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

The limitations are adequately addressed.

Reviewer jbje2024-08-11

Thanks for your response. In general, as a non-expert in the area, I found this paper quite inaccessible. I believe that improving the exposition and mathematical presentation will help a lot, as will adding definitions or at least references for IFT, GLLR, etc. In terms of Proposition 2.1 specifically, it is stated that it characterizes the unique optimal allocation, but it is very hard or me to connect this to the wall of math in the proposition. I don't think saying "The following statements are true" and then listing a sequence of technicality true statements is the clearest way to express a theorem. I would find more verbal explanation very helpful. Given that the authors have expressed a commitment to strengthening the exposition, and that other more expert reviewers seem to feel positively about the work, I am happy to increase my score from 3 to 5.

Authorsrebuttal2024-08-14

Thank you for increasing the score. We very much appreciate your comment that exposition and mathematical presentation needs to be improved. We are tweaking the exposition at a few places of the paper to improve the presentation. We illustrate the changes below through an updated version of Proposition 2.1: Proposition 2.1 is crucial for constructing the fluid dynamics. Proposition 2.2 provides a set of conditions which uniquely identify the optimal allocation $\omega^\star$. We define the quantity $N_{\min}=N_{1,1}+\sum_{a\in\bar{B}^c}N_a$. **Proposition 2.1:** For every positive $N$ satisfying $N\geq N_{\min}$, there is a unique set of variables $N_{\bar{B}}(N)=\left(N_a(N)\geq 0:a\in\bar{B}\right)$ and $I_B(N)$ satisfying the following conditions: $\sum_{a\neq 1}\frac{d(\mu_1,x_{1,a})}{d(\mu_a,x_{1,a})}=1$ where $x_{1,a}=\frac{N_1(N)\cdot\mu_1+N_a(N)\cdot \mu_a}{N_1(N)+N_a(N)}$ and $N_1(N)\cdot d(\mu_1,x_{1,a})+N_a(N)\cdot d(\mu_a,x_{1,a})=I_B(N)$ for every $a\in B$. Furthermore, $N_{\bar{B}}(\cdot)$ and $I_B(\cdot)$ are continuously differentiable w.r.t. $N$ for every $N>N_{\min}$. **Proposition 2.2:** Upon taking $B=[K]/{1}$ and $N=1$, $N_{\bar{B}}(1)$, as defined in Proposition 2.1, is same as the unique optimal allocation $\omega^\star$ solving the max-min problem in Eq.(1), and we also have $I_B(1)=T^\star(\mu)^{-1}$. Moreover, for every $N>0$, the unique solution $N_{\bar{B}}(N)=(N_a(N):a\in[K])$ satisfies $N_a(N)=N\omega_a^\star$. **[Providing references]** We have added references to the IFT in the revision. About the GLLR, we referred to the discussion in [13, Section 3.2] in Section 3 of the submitted version of our paper. Furthermore, in the revision, we have made a section in the appendix explaining the intuition behind GLLR.

Reviewer Aqmy2024-08-11

Thanks to the authors for addressing my questions and weaknesses. I think adding the specifications and explanations to the experiments and question 2 would make the paper more intuitive to understand. I intend to keep my score and good luck for the submission.

Authorsrebuttal2024-08-13

Thanks again for your suggestions. These will improve the paper's presentation.

Reviewer Uo1U2024-08-13

Response to Authors' Rebuttal

I thank the authors for their detailed and insightful response. All my questions have been fully addressed. I will maintain my current score.

Reviewer mata2024-08-13

Thank you for your response. After reading the rebuttal, I would like to maintain my original rating.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC