Thanks for the reviewer's comments.
**Q1**. …I suggest the authors add the corresponding results since existing methods can solve the studied imbalance problem partially.
**A1**: (1) Please note that our work focuses on optimizing AUC for federated learning. Therefore, we compared with strong and direct baselines of federated learning for optimizing AUC, including CoDA [1] and LocalSGDAM. As prior works [2, 3] have demonstrated that optimizing traditional cross-entropy (CE) loss will yield worse results than AUC maximization for imbalanced data, we tried not to include too many baselines that optimize CE loss in FL. (2) We would like to draw reviewer's attention that CoDA leverages the epoch-wise proximal term as in FedProx and the variance-reduction as in Scaffold for solving the min-max formulation of AUC maximization. Therefore, CoDA is much a stronger baseline than FedProx and Scaffold. (3) Nevertheless, we have conducted an experiment to compare with Scaffold and FedProx for optimizing CE loss on STL10 dataset with $p=4$, and the results are shown below. It can be observed that our algorithm can outperform those two baselines with a large margin.
| | Scaffold | FedProx | CoDA | LocalSCGDAM(Ours) |
|-----|----------|---------|-------|-------|
| AUC | 0.788 | 0.778 | 0.801 | 0.820 |
**Q2**. …why it can confirm the effectiveness of the proposed method?...
**A2**. The effectiveness of the proposed method can be explained below. (1) We optimize the compositional AUC formulation, which has been shown to learn much better feature representation than conventional AUC maximization [4]. (2) Our algorithm design and theoretical analysis ensure that our algorithm has a small sample complexity. This is also important as with the same number of epochs, an algorithm with a higher sample complexity may find a worse solution. Therefore, our theoretical analysis ensures that our algorithm can quickly find a good solution to compositional AUC maximization. Together, it can explain the effectiveness of our algorithm. (3) We would like to emphasize that it is not trivial to attain the $O(1/(K\epsilon^4))$ complexity for solving the compositional AUC loss as in our paper. For instance, [5, 6] show that the stochastic gradient descent (SGD) or stochastic compositional gradient descent (SCGD) algorithm under federated learning setting has the sample complexity of $O(1/\epsilon^{8})$ or $O(1/\epsilon^{5})$, which cannot match the sample complexity $O(1/(K\epsilon^{4}))$ of SGD for non-compositional optimization problems. In our theoretical analysis, we show that our algorithm LocalSCGDAM can achieve the sample complexity of $O(1/(K\epsilon^{4}))$, which can match that of traditional federated learning algorithms for non-compositional optimization problems.
[1] Guo et al., Communication-efficient distributed stochastic auc maximization with deep neural networks. ICML 2020.
[2] Yuan et al., Robust Deep AUC Maximization: A New Surrogate Loss and Empirical Studies on Medical Image Classification. ICCV 2021.
[3] Liu et al., Stochastic AUC Maximization with Deep Neural Networks. ICLR 2020.
[4] Yuan et al., Compositional Training for End-to-End Deep AUC Maximization. ICLR 2022.
[5] Huang et al., "Compositional federated learning: Applications in distributionally robust averaging and meta learning." arXiv preprint arXiv:2106.11264 (2021).
[6] Wang et al., "Memory-Based Optimization Methods for Model-Agnostic Meta-Learning and Personalized Federated Learning." JMLR 24 (2023): 1-46.