Thanks for your further response!
We hope that our response could address your concerns, and we are willing to provide further clarification if necessary.
**About repeated experiments and the comparison with UPS**
Table 1 reports the comparison results between CAP and the comparing methods. Due to the time limit, we select two strong competitors from the comparing methods in the original paper. According to your suggestion, we also include the SSL method, UPS, in the comparison. From the table, it can be observed that the performance of our method is significantly better than the comparing methods, especially when the number of labeled data is small. We will include these results and the discussion about UPS in the revised version.
Table 1. Mean and standard deviation of mAP over 5 runs (random seed = 1, 2, 3, 4, 5).
| | VOC | | | | COCO | | | |
| ---- | ------------ | ------------ | ------------ | ------------ | ------------ | ------------ | ------------ | ------------ |
| | p=0.05 | p=0.1 | p=0.15 | p=0.2 | p=0.05 | p=0.1 | p=0.15 | p=0.2 |
| CAP | 77.15 ± 0.58 | 82.54 ± 0.20 | 83.95 ± 0.24 | 85.04 ± 0.32 | 63.11 ± 0.35 | 67.96 ± 0.32 | 69.92 ± 0.41 | 71.23 ± 0.42 |
| IAT | 74.78 ± 0.82 | 81.11 ± 0.46 | 82.91 ± 0.30 | 84.38 ± 0.44 | 60.82 ± 0.26 | 66.13 ± 0.30 | 68.50 ± 0.32 | 69.94 ± 0.35 |
| PLC | 75.61 ± 0.67 | 81.58 ± 0.62 | 83.27 ± 0.47 | 84.48 ± 0.57 | 60.20 ± 0.18 | 65.60 ± 0.29 | 68.32 ± 0.35 | 69.85 ± 0.35 |
| UPS | 76.25 ± 1.41 | 81.23 ± 0.95 | 82.92 ± 0.67 | 83.71 ± 0.61 | 59.16 ± 0.31 | 64.54 ± 0.23 | 66.87 ± 0.22 | 68.29 ± 0.18 |
**About the label shift problem**
Many semi-supervised learning (SSL) methods have explicitly or implicitly adopted the assumption that the labeled and unlabeled examples follow the same distribution. For example, the early work [1] presented the expectation regularization, which encourages the the marginal distribution of model predictions on unlabeled data to match the marginal distribution of the ground-truth labels that was estimated during training based on the training examples. Many recent works [2-4] have applied this idea in deep semi-supervised learning. Compared with SSL that has achieved great advances, semi-supervised multi-label learning (SSMLL) (in the context of deep learning) is still in its nascent period of development. Therefore, our paper focuses on the standard setting where labeled and unlabeled examples follow the same distribution.
To show the influence of the label shift on the final performance of our method, following the works [5-6], we draw $P(y)$ from a Dirichlet distribution with concentration $\alpha$. We define the degree of label shift $\Delta=\sum_{k=1}^q|\hat\gamma_k-\gamma_k^*|$ between the labeled and unlabeled data. By using different values of $\alpha$, the degree of label shift $\Delta$ could be varied. From Table 1, we can see that the performance of CAP degrades slightly as the degree of label shift increases significantly. For example, when $p=0.1$ and $\alpha=1$ (the degree of label shift has increased by ten times), the performance of CAP is still better than the comparing methods in Table 1 in the paper.
We will add a more detailed discussion of the assumption in the revised version.
Table 1. MAP of CAP under different degrees of label shift on COCO. * denotes the result without introducing the label shift.
| $\alpha$ | \| | 1 | 10 | 100 | * | \| | $\alpha$ | 1 | 10 | 100 | * |
| :-------: | :----: | :---: | :---: | :---: | :-------: | :----: | :-------: | :---: | :---: | :---: | :-------: |
| $\Delta$ | **\|** | 1.345 | 0.904 | 0.940 | 0.136 | **\|** | $\Delta$ | 1.428 | 0.998 | 1.019 | 0.098 |
| $p$ = 0.1 | **\|** | 66.17 | 66.86 | 66.74 | **67.36** | **\|** | $p$ = 0.2 | 68.86 | 69.38 | 69.31 | **70.41** |
Recently, several methods have been developed specifically to solve the label shift problem [5, 6], where the class marginal distributions of the labeled and unlabeled data are different. This is really an interesting future direction of our work. To solve the label shift problem in the SSMLL scenario, a straightforward strategy is to estimate the class distribution of the unlabeled data using the well-established method [5, 6], and then apply CAP to obtain pseudo-labels.
[1] Simple, Robust, Scalable Semi-Supervised Learning via Expectation Regularization. ICML’07
[2] ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring. ICLR’20
[3] CoMatch: Semi-supervised Learning with Contrastive Graph Regularization. CVPR’20
[4] AdaMatch: A Unified Approach to Semi-Supervised Learning and Domain Adaptation. ICLR’22
[5] Detecting and Correcting for Label Shift with Black Box Predictors. ICML’18
[6] LTF: A Label Transformation Framework for Correcting Target Shift. ICML’20