Selective Sampling and Imitation Learning via Online Regression

We consider the problem of Imitation Learning (IL) by actively querying noisy expert for feedback. While imitation learning has been empirically successful, much of prior work assumes access to noiseless expert feedback which is not practical in many applications. In fact, when one only has access to noisy expert feedback, algorithms that rely on purely offline data (non-interactive IL) can be shown to need a prohibitively large number of samples to be successful. In contrast, in this work, we provide an interactive algorithm for IL that uses selective sampling to actively query the noisy expert for feedback. Our contributions are twofold: First, we provide a new selective sampling algorithm that works with general function classes and multiple actions, and obtains the best-known bounds for the regret and the number of queries. Next, we extend this analysis to the problem of IL with noisy expert feedback and provide a new IL algorithm that makes limited queries. Our algorithm for selective sampling leverages function approximation, and relies on an online regression oracle w.r.t.~the given model class to predict actions, and to decide whether to query the expert for its label. On the theoretical side, the regret bound of our algorithm is upper bounded by the regret of the online regression oracle, while the query complexity additionally depends on the eluder dimension of the model class. We complement this with a lower bound that demonstrates that our results are tight. We extend our selective sampling algorithm for IL with general function approximation and provide bounds on both the regret and the number of queries made to the noisy expert. A key novelty here is that our regret and query complexity bounds only depend on the number of times the optimal policy (and not the noisy expert, or the learner) go to states that have a small margin.

Paper

Similar papers

Peer review

Reviewer mVoN6/10 · confidence 3/52023-07-04

Summary

In this work, the authors make several contributions. Firstly, they propose a novel selective sampling algorithm (Algorithm 1) that utilizes a regression oracle and provide regret bounds and query complexity analysis. Secondly, they present a lower bound that demonstrates the unavoidable dependency of query complexity on the eluder dimension without additional assumptions. Thirdly, they introduce Algorithm 2, which is designed for to iid sample distributions and employs the disagreement coefficient as a substitute for the eluder dimension. For imitation learning, the authors prove an exponential gap in sample complexity between stochastic and deterministic expert demonstrations in the context of offline imitation learning. Lastly, they propose imitation learning algorithms that utilize regression oracles with selective sampling for both single and multiple stochastic expert settings.

Strengths

1.This work provides valuable insights into the field of neural network-based imitation learning with selective sampling, laying the foundations for future advancements in practical and efficient algorithms. 2.The authors' contributions encompass both selective sampling and imitation learning, showcasing the potential for designing imitation learning algorithms that offer provable efficiency. Their research opens up exciting possibilities for further advancements in the field.

Weaknesses

1.While this work presents self-contained proofs, it lacks a direct comparison of the imitation learning regret and query complexity of the proposed algorithm with Dagger and other existing algorithms. This omission makes it challenging to assess the efficiency and performance of the proposed approach accurately. 2.The experiment conducted on Cartpole may not provide sufficient insight and understanding of the algorithm's performance. Furthermore, the use of sophisticated selective sampling techniques may introduce significant computational overhead when applied to more complex experimental environments such as hopper and walker. This computational burden hinders the evaluation of the algorithm and makes it difficult to further explore its effectiveness in practical scenarios. 3.The usage of a deterministic Markov Decision Process (MDP) with a fixed starting state, which may not align with standard imitation learning problems. This restriction limits the applicability of the guarantees provided in this work to more general and diverse scenarios.

Questions

Questions: 1.Lemma 18 does not appear to me a variant of the performance difference lemma, where in the second inequality of the equation after line 1075 the value on the right is already $2H$, the upper bound of the performance difference. Please explain if this implies the other results that applies lemma 18 are already greater than $2H$. 2.In line 281, for AggreVate, I believe it takes $O(\log(H))$ rounds for the probability of executing expert policy to be lower than $1/H^2$, where by the union bound we have the probability of executing the learner's policy for whole $H$ steps greater than $(H-1)/H$. If mixing with the noisy expert is the only issue, I don't agree that AggreVaTe style algorithm should sample $\Omega(2^H)$ samples if I am allowed to mix the proposed algorithm 2 with noisy expert policy as AggreVaTe-style in the roll out. 3.I am confused why in line 266 we need a large margin over state space for Dagger instead of the recoverability assumption. By the performance difference lemma, we only need the expert's advantage function to be bounded by some value smaller than $H$ for Dagger to achieve better regret guarantee than behavior cloning. Please feel free to let me know if you need any reference. 4.I am curious why the number of queries for Dagger is not a straight line, which may due to sampling trajectories instead. Please provide more experimental details including model and training details for more transparent presentation. 5.Does the analysis in this work generalizable to non-deterministic learner policies? 6.Is proposition 1 extendable to stochastic transition dynamic setting? I am afraid the stochastic transition dynamics will make any roll out policy hard to reach the designed leaf node. 7.Is line 5 in algorithm 1 easy to compute when the policy class is exponentially large? 8.Querying all $M$ experts and assigning $H \times M$ learner policies is quite computationally demanding, any method to reduce to learning only one policy under some assumption of realizability? Minor suggestions: 1.In line 1006 it would be better to define $r(x,a) = 0$ for none leaf nodes. 2.In line 1022 use $ \forall \pi \in \Pi$ to avoid confusion. 3.In line 257, could you please clarify the definition of your regret, which is missing the expectation over transition dynamics. I assume you are in the deterministic transition setting. The state that would have been generated is a bit vague, please point out the deterministic MDP setting with unique starting state directly. 4.In line 256 'the learner proposes an action' is a bit vague, I suppose you are suggesting a deterministic learner's policy in stead of a stochastic random policy like a noisy expert. Could you please make it more clear by mentioning 'deterministic'? 5.In line 491, though using the last round policy as warm start may result in similar performance as training from random initialization. It would be beneficial to conduct one extra experiment comparing the result of using warm start and naively start from random initialization for each round, which would empirically justify this speedup. 6.In the Figure 2, Dagger takes around 13 iterations to converge, where it would be helpful to mark the exact number of annotations for Dagger to achieve near expert performance as a dotted line on the right of figure 2.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

2 fair

Contribution

3 good

Limitations

1.The lack of a direct theoretical comparison between the proposed algorithm's imitation learning regret and query complexity with Dagger and other preexisting algorithms hinders the interpretation of its efficiency. Comparative evaluations against existing algorithms would provide a clearer understanding of the algorithm's performance and effectiveness. 2.The experiment conducted solely on the Cartpole environment may not offer sufficient insights into the algorithm's behavior and capabilities. Additionally, the computational overhead associated with the sophisticated selective sampling technique might pose challenges when applying the algorithm to more complex environments like hopper and walker. This limitation makes it difficult to thoroughly evaluate the algorithm's performance in a broader range of scenarios. 3.The use of a deterministic Markov Decision Process (MDP) with a fixed starting state deviates from the standard settings in imitation learning problems. This restriction limits the guarantee and applicability of the proposed algorithm, as it may not generalize well to more diverse and realistic scenarios encountered in practice. 4.While Proposition 1 is supported by theoretical proof, it would be beneficial to include empirical observations to demonstrate its importance and practical relevance. Additionally, conducting experiments that involve behavior cloning would provide further insights and comparisons, enhancing the experimental evaluation and supporting the theoretical claims.

Reviewer hUJx7/10 · confidence 2/52023-07-05

Summary

The paper proposed a selective sampling scheme on an online regression oracle over a function class $\mathcal{F}$ and time horizon $T$ for multiclass setting, which has the regret guarantee $Reg^{\ell_{\phi} } (\mathcal{F}, T)$, and we assume the label $y_t$ is sampled through a link function $\phi$. The algorithm actively acquire a label when the margin (the difference among top two action scores) are small, which resembles sampling by disagreement in active learning research. Assuming the problem has a margin greater than some $\epsilon$ at all round and adversarially chosen context $x_t$, the sampling scheme achieve regret $\tilde{O}( Reg^{\ell_{\phi} } (\mathcal{F}, T) / \epsilon) $, and the labelling complexity as $\tilde{O} ( Reg^{\ell_{\phi} } (\mathcal{F}, T) / \epsilon^2 )$ ( also have dependence on eluder dimension of $\mathcal{F}$). In the case that the context $x_t$ is stochastically chosen, the algorithm achieves the same regret but fewer labels. The result can also be extended to imitation learning, where the regret bound and query number scales with episode number $H$ and teacher number $M$.

Strengths

The paper is well-written and proposed a simple sampling rule based on the margin on the fly. The appendix was well managed and easy to follow. The regret bound captures the difficulty of the problem and the regret of the online regression oracle, which is very general. The paper also shows lower bound on the tightness of the label complexity. The proposed sampling rule was derived for both adversarial and stochastically chosen context, showing the later is improved in terms of labelling complexity. The sampling rule, and analysis can also be extended to imitation learning. Experimental results shows the trade off between convergence speed and labelling complexity.

Weaknesses

Some possible extensions: 1. Since the experiment shows $\alpha$ in equation (7) was experimentally determined due to the complexity of finding the exact Lagrangian multiplier, whether theoretical bound can include the scenario: if $\Delta_t(x_t)$ can not be solved exactly. Instead we can only use an approximation denote as $\tilde{\Delta}_t(x) $ such that $| \tilde{\Delta}_t(x) - \Delta(x_t) | \le \delta $ for some threshold $\delta$. How does this error $\delta$ affects regret and sampling number. 2. For imitation learning multiple teacher, it looks like the regret and query number for Multiple teacher $M > 1$ (theorem 5) has a worse guarantee in terms of $M$ comparing to single teacher in theorem 4, due to $\sum_{m=1}^{M} $. This is counter intuitive in the sense that more resources was used. Whether the algorithm can be extended so we have regret and query number with $\min_{m \in [M] }$ instead of $\sum_{m=1}^{M}$. An example to illustrate current concern: in the single teacher setting, we use ${Oracle}_1$ has regret $R_1$. For the multiple teacher setting, we use ${Oracle}_1, {Oracle}_2, \cdots, {Oracle}_M$, which all have regret $R_1$, then multiple teacher setting is picking up $M R_1$ and single teacher is only $R_1$. Although they are using the same quality of teacher oracles.

Questions

see above

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The paper might have been limited by the submission page limit, which ends without an explicit discussion session. The paper already summarized its finding and significances. Currently limitation / improvement is addressed in the weakness session, a broader and domain specific limitation, future direction should be included if more space is permitted.

Reviewer A81c5/10 · confidence 3/52023-07-06

Summary

This paper studies selective sampling and imitation learning. In this setting a series of states arrives online, and a learner has the ability to choose to query expert labels/policy in order to correctly label the states/perform as well as an expert policy. The goal is to minimize the regret (i.e. the drop in performance relative to what the expert would achieve on the particular sequence) while also making as few queries to the expert labels/policy. The authors design algorithms under the additional assumption that there is black-box access to an online regression oracle for the class of functions $\mathcal{F}$ under consideration. The algorithms achieve regret bounds that depend linearly on the regret achieved by the online regression oracle, plus the number of times that the expert-provided label function has a small-margin in the online sequence. These regret bounds are essentially best-possible considering the performance of the regression oracle. For the number of expert queries, the bounds for the algorithms grow proportional the product of the regret bounds of the oracle and the eluder dimension. The authors further show that the dependence on Eluder dimension is necessary.

Strengths

The paper provides a clear general method to transform an online-regression oracle for a given function class $\mathcal{F}$ into an algorithm for active learning using the class $\mathcal{F}$. This method essentially maintains the regret of the original regression oracle (which has access to all the labels), while making as few oracle queries to the expert labels/policy as possible (without making further assumptions).

Weaknesses

1. Both algorithms in the paper rely on solving an optimization problem (Equation (5) for Algorithm 1 and Equation (6) for Algorithm 2). It is not immediately clear how to use the online regression oracle to solve these problems. If they cannot be solved via the oracle, it seems likely that they are generally intractable for many function classes $\mathcal{F}$ of interest. This seems like an important issue for both algorithms, and significantly limits the utility of the method if an additional oracle is required to solve the problems in (5) and (6). 2. A minor point: the notation on line 9 of Algorithm 1 was initially confusing to me. Presumably the regression oracle is called with all previously queried labels, but the notation actually used was never introduced and so it is unclear what was actually meant.

Questions

1. I don't immediately see why the optimization problem in Algorithm 1 Equation (5) is tractable, even given an online regression oracle for the function class $\mathcal{F}$. Am I missing something or is another assumption needed here (e.g. that we have a second oracle to solve the optimization problem in (5))? Alleviating my concerns about this issue would cause me to increase my score. 2. Can you clarify exactly which sequence is sent to the oracle in line 9 of algorithm 1?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

Yes.

Reviewer hxAC6/10 · confidence 2/52023-07-26

Summary

The paper studies algorithms for selective sampling with applications to imitation learning. In the former setting the authors propose an algorithm build around an online regression oracle and derive bounds on the overall regret and number of queries. These are provided for both a general (potentially adversarial) setting as well as a potentially more favourable stochastic setting. A key component is the dependence on the eluder dimension of the regressor family and the authors show that a dependency on this is unavoidable by also providing matching lower bounds. In the imitation learning setting the notion of an expert confidence *margin* is introduced. The authors argue that in an offline setting requirements for expert trajectories can grow exponentially w.r.t. to the length of the considered time horizon and consequently an online learning setup is required to achieve reasonable regret. Based on the algorithm from the selective sampling setting a matching formulation for imitation learning with corresponding regret derivations is presented. Finally, the notion of expert uncertainty encoded in margin scores allows for an extension to a multi-expert setting.

Strengths

The paper is well structured and clearly elucidates the short-comings of prior work it addresses. The extensions proposed to overcome them are clearly motivated and the proposed algorithms appear sound.

Weaknesses

Despite being motivated by making the algorithms more fit for practical applications, there is a rather limited experimental evaluation of the proposed algorithms. It would be interesting to compare their empirical performance to existing baselines such as DAgger.

Questions

-

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

-

Authorsrebuttal2023-08-10

Rebuttal Continued

### All results trivially extend to Stochastic MDPs All our results can be trivially extended to stochastic MDPs. We provide more discussion and proof in Appendix D.4 on Page 45. For ease of analysis and presentation, we consider a scenario where the learner interacts with deterministic dynamics $\\{T\_{t, h}\\}\_{h \leq H}$ at time $t$ that could be adversarially chosen (and is unknown to the learner). However, because the dynamics can change adversarially, our assumptions are strictly weaker than assuming stochastic dynamics. In particular, in order to simulate stochastic dynamics, one can simply consider that for every round $t \in [T]$ a random seed $z\_t$ is chosen from a fixed unknown distribution. Given the seed $z\_t$, the adversarially chosen dynamics can simply be set as $\\{T\_{t, h}\\}\_{h \leq H} = T(; z\_t)$ where $T$ is some stochastic dynamics. ### Is proposition 1 extendable to a stochastic transition dynamic setting? The main contribution of the lower bound was to provide a lower bound for prior works on non-interactive imitation learning *when the expert provides noisy feedback*. On the other hand, our algorithm can solve the problem instance in the lower bound after collecting polynomially many samples. Since it is a lower bound, we only provided one instance which was a determinstic MDP. However, we believe that a similar lower bound could also be constructed for stochastic MDPs by changing the reward function on the leaf nodes. Note that a simpler stochastic MDP construction may be obtained by directly using a combination lock with rich observations as the underlying MDP. ### Does the analysis in this work generalizable to non-deterministic learner policies? Note that the policy $\pi^\star$ that we wish to compare to in our regret bounds is deterministic. Thus, we do not require stochasticity in the learner policies in order to obtain the optimal regret/query complexity bounds. However, our algorithm can be extended to work with stochastic policies parameterizations (e.g. neural networks with softmax) when they are sufficiently capable of representing close-to-deterministic policies (e.g. by tuning the scale parameter in softmax).

Reviewer hUJx2023-08-13

Dear Authors, Thank you for your several detailed examples on solving the label query parameter $\Delta_t(x_t)$, for elaborating on how $\delta$ error propagates. Thank you for explaining the dependence on $M$ for multiple expert setting. On top of the interesting theoretical bounds being presented given access to an oracle, my concerns has been addressed, hence I increased the score from 6 to 7.

Reviewer A81c2023-08-16

Thank you for your explanation, particularly of how to efficiently implement the optimization in equation (5) given access to a regression oracle. This would definitely be good to include in future versions of the paper. I will increase my score to 5.

Authorsrebuttal2023-08-16

Will do!

Thank you for your positive response to our rebuttal! We will definitely include a section on computationally efficient implementations in the revised version of our paper.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC