Group-wise oracle-efficient algorithms for online multi-group learning

We study the problem of online multi-group learning, a learning model in which an online learner must simultaneously achieve small prediction regret on a large collection of (possibly overlapping) subsequences corresponding to a family of groups. Groups are subsets of the context space, and in fairness applications, they may correspond to subpopulations defined by expressive functions of demographic attributes. In contrast to previous work on this learning model, we consider scenarios in which the family of groups is too large to explicitly enumerate, and hence we seek algorithms that only access groups via an optimization oracle. In this paper, we design such oracle-efficient algorithms with sublinear regret under a variety of settings, including: (i) the i.i.d. setting, (ii) the adversarial setting with smoothed context distributions, and (iii) the adversarial transductive setting.

Paper

References (37)

Scroll for more · 25 remaining

Similar papers

Peer review

Reviewer TY8p7/10 · confidence 3/52024-07-07

Summary

Multi-group learning has been attracted attention and solutions for small-sized groups are already available. This paper further studies the online learning case when the group $\mathcal{G}$ is large or even possibly infinite. In addition to designing an algorithm, the paper also provides extensive theoretical analysis on the results in three settings: the i.i.d. setting, the adversarial setting with smoothed context distributions, and the adversarial transductive setting.

Strengths

1. The paper studies an interesting and important problem. The size of possible groups may indeed be very large in real cases, and infinite as the sample size infinitely grows. 2. The paper, while using relatively advanced mathematics, tackles the problem with well-known and intuitive techniques in optimization and game theory, and consequently achieves satisfying theoretical results. 3. The presentation is very readable: the definitions, results, and remarks are presented in a nice order and achieves the level of rigor for a theoretical paper. Overall I am in favor of this work because of the elegant solutions for an important problem.

Weaknesses

1. The theoretical analysis is restricted on binary case, i.e. $\mathcal{Y} = \{ -1, 1 \}$. I am aware that the multi-class case would be much difficult to analyze, but this certainly shall be one of the possible directions in the future. 2. (minor) Definition 4.1 uses measure theory that is in a graduate-level course in real analysis and is likely a rarely appearing concept for the majority audience. To keep the paper self-contained, it might be helpful to thoroughly define all concepts such as essential supremum and the differentiation of measures.

Questions

I do not have any questions at this moment.

Rating

7

Confidence

3

Soundness

3

Presentation

4

Contribution

4

Limitations

Yes.

Reviewer sHAP6/10 · confidence 4/52024-07-12

Summary

This paper studies the problem of online multi-group learning where the algorithm should achieve sublinear regret with respect to all groups. They propose a group-wise oracle-efficient algorithm that avoids enumerating all groups by accessing the group oracle. An $\tilde{O}(\sqrt{dT/\sigma})$ regret bound is derived for $\sigma$-smoothed online learning setting.

Strengths

1. The paper addresses the problem of avoiding enumeration of groups, which improves the efficiency of the algorithm when the space of group is large. 2. The reduction from online multi-group learning to the AMF framework is novel and interesting.

Weaknesses

1. The discussion of related work for online multi-group learning is not adequate. It would be better if the author included the regret bound and online learning setting in the related work. I think listing a table for comparison among these works is better. 2. The motivation of using the AMF framework and FTPL is still not clear and needs more discussion. Moreover, it would be better if the author could clarify the novelty in the analysis. 3. It would be better if the author could also show the lower bound of this problem. Thus we can see how tight regret is obtained and which part can be improved in future work. 4. The computational efficiency of the $(G, H)$-optimization oracle is not discussed. It seems that this oracle is much harder than the ERM problem in [Ach23+] since this oracle optimizes over both $G$ and $H$.

Questions

Please see the weaknesses part.

Rating

6

Confidence

4

Soundness

3

Presentation

2

Contribution

2

Limitations

The authors have addressed the limitations and societal impact.

Reviewer wskr7/10 · confidence 5/52024-07-12

Summary

The paper provides oracle efficient algorithms for online multi-group learning. The goal is to obtain sublinear regret for each group subsequence $g \in G$ simultaneously. [BL20] showed that $o(T_g)$ regret is possible with finite $H$ and $G$ but must enumerate both $H$ and $G$. [Ach+23] showed $o(T_g)$ regret while being oracle efficient in the hypothesis class, but must enumerate over $G$. In this work, the authors use an optimization oracle for $G \times H$ jointly and avoid explicit enumeration for either G or H. To do so, they reduce the multigroup online learning to Adversary moves first framework of [Lee+22], and adapt results from oracle efficient online learning in the standard (no-groups) setting.

Strengths

- The paper is technically solid and a substantial contribution to online learning with multi group regret gurantees, and is the first to be oracle efficient in both G and H. - The reduction of multi group regret to AMF regret and the resulting minmax game simplification is novel. - They provide multigroup regret analogs of smoothed online learning, generalized FTPL.

Weaknesses

- For the $G \times H$ oracle mentioned in definition 2.2 the authors defer to [GKR22] who in turn provide two constructions: one based on ternary classification and another alternating minimization heuristic. It would be beneficial to add a longer discussion on this in the main body or the appendix to make the paper self contained. - Computational efficiency of the $G \times H$ oracle, this hasn't been discussed explicitly, are there group structures for which the optimization over G is efficient and doesn't involve enumeration?Algorithm 6 in [GKR22], doesn't discuss this either, it seems like without any further assumptions, enumeration is the best we can do in practice.

Questions

Please address the above. Also, the appendix describes multi-class action space, what would change with real valued labels?

Rating

7

Confidence

5

Soundness

4

Presentation

4

Contribution

3

Limitations

The authors discuss limitations of their paper.

Reviewer K3De6/10 · confidence 3/52024-07-18

Summary

This paper studies the online multi-group learning problem, and considers it in both smoothed context and general ($\gamma$-approximatable) settings. Unlike the traditional online learning setting, this paper considers achieve low regrets for all groups (subsequences). It utilizes the adversary moves first (AMF) framework for multi-objective online learning from [Lee+22] and the follow-the-perturbed-leader (FTPL) algorithm to handle the large number of groups. Through subtle algorithmic design and analysis, the paper achieved computationally-efficient and oracle-efficient learning.

Strengths

The paper studied the multi-group online learning problem, which is related to fair learning and subgroup robustness literature, both are significant. Under the smoothed setting, it achieves $\tilde{O}(\sqrt{dT/\sigma})$ regret; while under the general setting where $\gamma$-approximability is guaranteed, it achieves $\tilde{O}(\sqrt{NT_{g}\log\lvert H\rvert \lvert G\rvert})$ for each group $g$. It first considers the i.i.d setting (both offline and online) then quickly moves forward to the smoothed online setting. Under the AMF framework, it considers two players: $(G,H)$-player (adversary that has access to an optimization oracle $\text{OPT}^{\alpha}_{(G,H)}$ that maximizes the learner's loss) and the $H$-player. Under such a game-theoretic perspective, the paper proposes algorithms that achieves bounded regret. It then adapts the algorithm to broader settings (non-smoothed settings), where the $\gamma$-approximability is assumed. For groups with different densities, the algorithm can achieve low regret for each group $g$. The algorithmic designs are subtle and the analysis is sound to the review.

Weaknesses

The main text in Section 2 and after are fine, while Section 1 needs improvement. Specifically, some descriptions are unclear. For example, the bound $\tilde{O}(\sqrt{NT_{g}\log\lvert H\rvert \lvert G\rvert})$ is achieved for each group $g$, but this is not brought forward in summary of result (Line 76) and the last time $g$ is mentioned as one group is in Line 18. Line 54-55, if $G$ is rich and expressive, then all subgroups well-approximated by a group $g$ in $G$ will be "covered" by whom? What does this sentence mean?

Questions

See Weaknesses.

Rating

6

Confidence

3

Soundness

3

Presentation

2

Contribution

3

Limitations

No concerns.

Reviewer wskr2024-08-11

Thanks for the addressing my comments, I stand with my original assessment for accepting this paper. I appreciate the discussion on oracles for GxH and highlighting this in the main body and providing some concrete though not always computationally efficient heuristic in the appendix would be great, these should also address the other reviewer's concern.

Reviewer K3De2024-08-11

Official Comment by Reviewer K3De

Thank you very much for the response. Please include the discussions in revision for better readability. I will keep the current review.

Reviewer sHAP2024-08-12

Thanks for your response. You have addressed my concerns about the related work, lower bound, and computational efficiency of the oracle. Hence, I will raise my score to 6.

Reviewer TY8p2024-08-13

Thanks for the response

I appreciate the concise response by the authors. I am happy to keep my score.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC