A Competitive Algorithm for Agnostic Active Learning

For some hypothesis classes and input distributions, active agnostic learning needs exponentially fewer samples than passive learning; for other classes and distributions, it offers little to no improvement. The most popular algorithms for agnostic active learning express their performance in terms of a parameter called the disagreement coefficient, but it is known that these algorithms are inefficient on some inputs. We take a different approach to agnostic active learning, getting an algorithm that is competitive with the optimal algorithm for any binary hypothesis class $H$ and distribution $D_X$ over $X$. In particular, if any algorithm can use $m^*$ queries to get $O(η)$ error, then our algorithm uses $O(m^* \log |H|)$ queries to get $O(η)$ error. Our algorithm lies in the vein of the splitting-based approach of Dasgupta [2004], which gets a similar result for the realizable ($η= 0$) setting. We also show that it is NP-hard to do better than our algorithm's $O(\log |H|)$ overhead in general.

Paper

Similar papers

Peer review

Reviewer orr47/10 · confidence 3/52023-06-25

Summary

This paper studies agnostic active learning with a finite hypothesis space. The goal is to achieve a competitive query complexity with an optimal algorithm.

Strengths

Interesting and important problem. The approximation hardness based on Set-Cover is a nice addition. --- rebuttal comment --- After reviewers clarified concerns and added more results, I raised my score from 3 to 7.

Weaknesses

The authors seem to be unaware of "Minimax analysis of active learning" by Steve Hanneke and Liu Yang [JMLR 2015]. Hanneke and Yang introduce the star number (a combinatorial dimension), which characterizes various active learning settings (realizable, different noise settings, including the one here) leading to near-tight lower and upper bounds. This seems to make the analyses here a special case of the achieved results. E.g., on a quick look Theorem 8 seems to have a similar competitive ratio as the proposed algorithms: roughly $d\log (1/\varepsilon)$ (with $d = VC$) instead of $\log |H| \geq d$. Similarly, the discussed bounds $\mathcal{O}(m\log H)$ have been devised by Hegedűs, Tibor. "Generalized teaching dimensions and the query complexity of learning." [COLT 1995] long before Dasgupta and Nowak. Claiming polynomial runtime is misleading, as one typically would mean polynomial in the instance space $X$ (or the sample, like in standard PAC) and not the hypothesis space $\mathcal{H}$, as is here the case.

Questions

The authors write $|H|$ multiple times implying that the hypothesis space is finite but also use packings / coverings of $H$. Do the results generalize to hypothesis spaces with infinite size?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

1 poor

Limitations

See weaknesses.

Reviewer YT5S7/10 · confidence 3/52023-06-28

Summary

This paper applied a multiplicative weight update / generalized binary search style algorithm to solve agnostic active learning. It proposes a novel "capping" approach to the weight (over hypotheses) to ensure the potential function always grows by some amount, and it proves this amount is lower bounded by $\Omega(1/m^*)$ where $m^*$ is the theoretical optimal instance-dependent label complexity. In the end, it shows that the proposed algorithm is competitive in the sense that its label complexity is at most, roughly speaking, $\tilde{O}(m^* \log |H|)$. And it also shows that removing the extra "log(|H|)" factor is NP-hard. -- I've read the author's response. It addressed my major concern about related work so I increased my score to 7. I would encourage the authors to provide a comprehensive comparison with related works, including directly comparing bounds in different settings. I'm still not very convinced about the argument around computational efficiency, but that should not block this paper from being accepted.

Strengths

- The proposed algorithm is a novel and nontrivial modification of the classical multiplicative weight update algorithm. The analysis from a competitive perspective also looks new to me in the setting of agnostic active learning. - I briefly checked the proof and they look sound to me. - Understanding the theoretical lower and upper bounds of instance dependent label complexity bounds for agnostic active learning is an interesting problem, and this paper makes a solid contribution by providing an algorithm whose label complexity is close to the optimal with an additional log(|H|) factor.

Weaknesses

1. The additional factor of log(|H|) makes the label complexity of the proposed algorithm look not very strong to me: it can be worse than existing algorithms in many scenarios. The argument that getting rid of this factor is NP-hard does not look very relevant to me, since the main contribution of this paper is more on the information-theoretic side (the proposed algorithm is already by no means computationally efficient in the sense that it is polynomial in the number of hypotheses (or its epsilon cover at best) and the size of example space X), and it is known that even learning linear classifier without too much assumptions on noise is NP-hard. 2. It would clearer if there were more discussion about how good the label complexity is by comparing it with known upper and lower bounds in different settings. Especially, there are a few papers (e.g. Hanneke, Steve, and Liu Yang. "Minimax analysis of active learning." J. Mach. Learn. Res. 16.1 (2015): 3487-3602.) that give minimax analysis of active learning. I would like to see how your results compared with theirs. It would also be interesting if you could mention how the proposed algorithm works with noise assumptions (bounded, Tsybakov, etc.). 3. This paper focuses on the case where $\epsilon \gtrapprox \eta$, but it looks to me the case where $\epsilon$ is smaller can also be interesting (though of course the label complexity won't be exponentially better w.r.t. 1/\epsilon). It would be interesting if the theoretical results could cover that as well. 4. This paper is overall clear, but some wordings are a bit confusing to me unless I check the detailed proofs. E.g., the proof sketch in line 149 - 154 ("algorithm", "majority label", "inputs", "independent").

Questions

I would like the authors to comment on the weaknesses 1~3 I mentioned above.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

n/a

Reviewer LeWc7/10 · confidence 4/52023-07-07

Summary

The paper studies agnostic active learning by proposing a competitive algorithm that achieves at most a $\log H$ multiplicative factor on top of the optimal query lower bound of $m^*$. While similar result was known for the realizable setting, this paper makes a step toward understanding the agnostic setting. As a complementary result, it also shows that it is NP-hard to improve the $O(\log H)$ overhead both in realizable and agnostic settings.

Strengths

The paper proposed a strategy that actively learns the target function in the agnostic setting with competitive sample complexity. Given that similar strategies only handle the realizable setting, it makes a significant contribution to the active learning problems in general settings. The proofs seem sound. The paper is well written.

Weaknesses

The paper is clear in the description of the algorithm and the analysis of how it queries. However, the computational cost of the algorithm is vague. In Theorem 1.1, it claims that the algorithm runs in polynomial time, but it seems not so at least in the dimension parameter d.

Questions

Can the author comment on the exact running time of the algorithm on each parameter, for example, $d, m, |H|, |X|, \frac{1}{\epsilon}$, (or any other important parameters)?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

4 excellent

Contribution

3 good

Limitations

No concerns.

Reviewer wAoo4/10 · confidence 3/52023-07-10

Summary

The authors provide an algorithm for learning in the presence of agnostic noise. The algorithm finds a hypothesis that gets error $O(\eta)$ where $\eta$ is the noise parameter. The algorithm requires querying specific points, so it uses a stronger oracle than the standard active learning. The algorithm is possibly inefficient for many classes as it is polynomial in the number of hypotheses (or to the size of the cover). Their algorithm is optimal up to log factors, and they provide a lower bound showing that it is NP-Hard to improve this.

Strengths

This work provides a query-efficient algorithm with tight results (upper and lower bound) for the agnostic setting.

Weaknesses

The setting in their analysis is not active learning but learning with queries which is a stronger oracle. So the title of the paper/abstract is misleading. In my opinion, active learning is a completely different model than the one the authors use. Also for examples the 41-46 lines, the papers there are for active learning and not for the model the authors analyze. I think the authors should be more clear about this. About the NP-Hardness result. This result makes sense when the $|H|$ is polynomial over $\epsilon$ and the dimension. But almost always the $|H|$ is exponentially large, i.e., $d$-dimensional functions.

Questions

N/A

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

2 fair

Contribution

2 fair

Limitations

No liomitations.

Reviewer orr42023-08-14

Good points, you are right, Hanneke and Yang's star number and disagreement coefficient in general do not seem to yield the desired results, since they take a worst-case distribution (and potentially sample). However, your given example is in the finite and realizable case. In such a case the extended teaching dimension $\text{EXT-TD}(H)$ guarantees near-tight sample-dependent bounds. $\text{EXT-TD}(H)$ is a lower bound and $O(\text{EXT-TD}\log|H|)$ as an upper bound, see Hegedűs [1995], exactly the $O(\log|H|)$ guarantee you state. Subsequently, Hanneke achieved similar distribution dependent bounds by adapting the extended teaching dimension to the agnostic case. E.g., Theorem 4 in Hanneke [2007] seems to yield a similar results to you. Please clarify, if there is some misunderstanding. I just want to understand to what extend the achieved results are novel. Another quick question: Your main algorithm seem to require to know $m^*$ (the optimal number of samples), which is of course not true in general. Your bounds are only interesting if $m^*$ is unknown. But I guess you can apply some sort of doubling trick? Thanks! Best, orr4 Hegedűs, Tibor. "Generalized teaching dimensions and the query complexity of learning" COLT 1995 Hanneke, Steve "Teaching Dimension and the Complexity of Active Learning" COLT 2007

Authorsrebuttal2023-08-15

For sure, the example above is in the realizable, exact case which has been solved for decades. The point of our paper is to get similar bounds for the agnostic case. But the prior work for the agnostic case can't even solve the example above. Hanneke '07 ----- Hanneke '07 fails both algorithmically and analytically on the same example, for $\eta = \epsilon = 3/N$. You point specifically to Theorem 4 of Hanneke '07, but this has exactly the same issue as Hanneke-Yang '15: the lower bound is for the worst-case distribution over $X$. The relevant line of Theorem 4 of Hanneke '07 is: $$ XPTD(V,(1-\epsilon)/\epsilon, \delta) \leq \sup_{D' \in \mathcal{D}} LQ(\mathbb{C}, D', \epsilon, \delta, 0) $$ which involves a supremum over all distributions $D'$ supported on $\mathcal{X}$. So there *exists* a distribution over the support such that the upper and lower bounds are both related to teaching dimension, but it probably isn't the given distribution. (The bound is also worse in that the lower bound is *partial* teaching dimension, which can be much smaller, particularly if $\delta$ is reasonably large.) And it's not just an analytical issue, if you read through the Algorithm ReduceAndLabel in Hanneke '07. Consider the same example as above, but with $\eta = \epsilon = \frac{3}{N}$. The optimal algorithm is still $O(\log N)$ queries: just query the last $\lg N$ bits and output that hypothesis, which might be slightly wrong in the unary region but has error $\eta + \frac{2}{N}$. In Hanneke '07, the first step is to take a sample of $\frac{X}{5 \log |H|}$ examples; in the example above, this is probably going to be entirely in the unary region. Then the algorithm calls Reduce on this sample, which repeatedly takes $O(1/\eta)$-size subsamples and labels their minimal specifying set; but if the original sample is unary, this set will be of size $O(1/\eta)$. So Hanneke '07 will use a linear number of queries on this example. Wait, you might say, what about Theorem 3 of Hanneke '07? That's an upper bound in terms of extended teaching dimension that doesn't involve a supremum over distributions. But it's analyzing the algorithm that we know fails on this example, so of course it has to be a weak upper bound. Why? The issue is that the distributional version of extended teaching dimension considered in Hanneke '07 isn't terribly good. It's in terms of a parameter $n$, which is set to about $\frac{1}{16 \eta}$; the distributional extended teaching dimension basically says: if you only had $n$ unlabeled data points from $\mathcal{D}$, how many labels would you need to teach? And in the example, with only $O(N)$ unlabeled data points you will probably still be missing a constant fraction of the binary points, so the teaching dimension remains polynomially high. If you have access to $\gg \frac{1}{\eta}$ unlabeled samples, you can hope to do much better. [*] It's a minor point, but I think there's an error in the proof of Theorem 3 in Hanneke '07 that slightly affects exactly what polynomial dependence it gets on this example. I think the actual algorithm takes $\tilde{\Omega}(N)$ queries, but the theorem statement would give an $N^{0.99}$ upper bound because you would see a constant fraction of those binary points. The algorithm samples $U_i$ from $U$, then subsamples that *with replacement* and finds a minimal specifying set. The distributional extended teaching dimension bounds the size of the specifying set whp for a *single round* of subsampling, but the double-round subsampling is not exactly the same distribution. The proof of Lemma 3 doesn't do anything to address this issue (it just says "For each, we make at most $t$ label requests with probability $\geq 1 - \frac{\delta}{2s}$", which would be true by assumption for the single round sampling), and in fact on the example with $\eta = \epsilon = 3/N$ it matters a bit. But regardless of whether the bound is $N$ or $N^{0.99}$, it's not the logarithmic dependence of optimal or our algorithm. On the need for knowing $m^*$ ----- You're right that our algorithm is written to assume $m^\*$, but it's a pretty minor dependence that can be removed. Thanks for pointing it out. The algorithm only uses $m^\*$ to set the number of iterations $k$ in the first stage. One could modify the algorithm to remove this dependence, as follows: $m^\*$ enters the proof through Lemma 2.1, which combined with (1) gives a lower bound on the expected potential gain in each round. But the quantity being bounded (LHS of Lemma 2.1 / RHS of (1)) does not depend on $h^\*$ or $m^\*$, and can be computed by the algorithm. So the algorithm *knows* a lower bound $\tau_i \gtrsim \frac{1}{m^\*}$ on how much the potential is expected to grow in each round $i$. Then the algorithm can stop as soon as $\sum_{i=1}^k \tau_i \geq O(\log H)$, at which point the potential is positive whp and the second stage will succeed.

Reviewer orr42023-08-17

Thank you for the clarifications. I now think that this paper is valuable to the active learning community at NeurIPS and hence changed my vote to accept.

Reviewer YT5S2023-08-15

Thanks for the response!

I'm satisfied with the response about "Relation to Hanneke-Yang '15", but I would like to see your comment to the point about the extended teaching dimension raised by reviewer orr4. I have a few follow up questions: - Re: "On the relevance of time complexity": It is still unclear to me why "an exponential time optimal algorithm would be trivial by brute force". In particular, we want an algorithm with low label complexity w.r.t. an **unknown** distribution, I don't see how brute force could deal with that. Is there any reference to this aspect? - Could you comment on my point that "the additional factor of log(|H|) makes the label complexity of the proposed algorithm look not very strong"?

Authorsrebuttal2023-08-15

See our response to orr4 about the distributional extended teaching dimension result of Hanneke '07. Q1: We want a low label complexity w.r.t. an unknown distribution over $Y \mid X$, relative to the optimal label complexity over all distributions over $Y \mid X$. So for every algorithm, we can determine its performance on every distribution over $Y \mid X$, then take the algorithm with minimax complexity. The running time is horrendous, but it's computable to arbitrarily good approximation. Q2: Our NP hardness result shows we could not hope for a polynomial time algorithm that avoids the $\log |H|$ factor. It's the best one can do even in the realizable, exact setting. Moreover: 1. Competing algorithms (e.g. by Hanneke) cannot even achieve that factor. 2. There are interesting cases, like noisy binary search, where it isn't too big. 3. There are cases (also like noisy binary search) where the algorithm outperforms the generic bound.

Reviewer LeWc2023-08-20

Official Comment by Reviewer LeWc

Thank you very much for your response. It is very clear to me now. I encourage the authors to add these comments in their revision. My review and score remain unchanged.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC