A Graph-Theoretic Framework for Understanding Open-World Semi-Supervised Learning

Open-world semi-supervised learning aims at inferring both known and novel classes in unlabeled data, by harnessing prior knowledge from a labeled set with known classes. Despite its importance, there is a lack of theoretical foundations for this problem. This paper bridges the gap by formalizing a graph-theoretic framework tailored for the open-world setting, where the clustering can be theoretically characterized by graph factorization. Our graph-theoretic framework illuminates practical algorithms and provides guarantees. In particular, based on our graph formulation, we apply the algorithm called Spectral Open-world Representation Learning (SORL), and show that minimizing our loss is equivalent to performing spectral decomposition on the graph. Such equivalence allows us to derive a provable error bound on the clustering performance for both known and novel classes, and analyze rigorously when labeled data helps. Empirically, SORL can match or outperform several strong baselines on common benchmark datasets, which is appealing for practical usage while enjoying theoretical guarantees.

Paper

References (86)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer 1J6p6/10 · confidence 3/52023-07-04

Summary

The authors describe a theoretical framework for contrastive learning of representations in an open-world setting, where both labelled data and unlabelled data of potentially new classes is available. They explicitly describe the graph encoding positive sample connections and formulate a contrastive loss that amounts to a spectral decomposition of the adjacency matrix. Using this framework, they prove two theorems about the change in clustering quality of their representations via k-means when incorporating some labels. The main take-away is that labelled data helps to cluster another class if this other class has strong connections to the labelled class via data augmentation and was previously poorly clustered. Their method also outperforms several competitors for open-world and self-supervised representation learning with subsequent clustering.

Strengths

I appreciate the theoretical nature of the paper. Understanding the behavior of machine learning models when a clear learning signal via labels is unavailable or just partially available is important for their development and trustworthiness. It is also encouraging that their framework does not only allow a formal investigation but also holds up practically. I find the paper is well written with an appropriate deferral of more technical parts to the appendix. The authors also discuss connections of their work to the more common contrastive SimCLR loss. Their theory seems to be novel for the setting of open-world representation learning.

Weaknesses

**Strength of different types of augmentations:** In the illustrative example in section 4.1. it is assumed that the probability $\tau_c$ of augmenting across classes if there is a shared attribute (e.g. augmenting a blue cube to a blue ball) is larger than the probability $\tau_s$ of just changing attributes within a class (blue ball to red ball). I find this quite counterintuitive. First, I am surprised that $\tau_c$ should be non-zero. This might be achievable in the toy setting, but for any real world data, say CIFAR10, it is very unlikely, if not impossible, to augment two images of different classes to the same image. Maybe a plausible change of class can be achieved via cropping (cropping an image of a dog in front of flowers to just the flower) but it still seems unlikely that there should be another image in the dataset that augments to the same view. The same argument applies, to a lesser extend, also to the question if two images of the same class can be augmented to the same view (positivity of $\tau_s$). In the toy setting, I also wonder how positions and sizes were treated. Can a large blue ball in the top left corner only augment to a large blue cube in the top left corner or to any blue cube? Second, I am surprised by the order of augmentation probabilities. Is it not more likely for two data points of the same class to augment to the same view than for two data points of different classes? A similar requirement seems to relevant for Theorem 4.2 (l255 ff). How crucial are these assumptions (or corresponding ones) for the outcome of the illustrative example, the main theory, and the empirical validation? How relevant does this make the theoretical analysis in real settings? **Empirical illustration of the main theorems:** It would strengthen the paper if the authors had included experiments that showed how the predicted quantities in Thm 4.1 and 4.2 behave empirically in real / toy settings. In particular, a graph that shows the $\Delta(\delta)$ terms in both theorems for different $\delta$ values both according to their theory and in terms of actual experiments. This would also make it more evident how restrictive the assumptions of the theorems are and how tight the inequality is. The size and evolution of the three summands in the last equation of Thm 4.2 would also be interesting. **Minor:** - I think that highlighting which feature is more separable in Fig 3b) is a bit problematic. First, UMAP plots are known to artificially tear apart structures. Second, the shape separability in 3b1) is also very good (but admittedly less obvious when looking at the plot as the color of the marker is much more prominent that its shape in the 2D plot). Therefore, I would recommend to rather stress the fact that when incorporating labels, the islands of the same shape but of different color seem to be much closer together (i.e. much less separable by color) than without labels. - It would have helped my understanding of the paper if it was stated more explicitly that label information is made available by providing the label of existing but previously unlabelled points as opposed to adding new labelled points. - I would suggest to also assign the bottom matrix in Fig 2b) as $\eta_l A^{(l)}$ as in the line above and then add another line $A=...$ below.

Questions

**Size of the data augmentation graph:** The node set of the graph described in 3.1 is the set of all augmented views of all available data. Why is this even finite? Consider, for instance, image data. Are there not infinitely many ways to color jitter even a single image? Or is the idea to only consider a fixed and finite set of augmentations of each data point that is selected prior to training? This would be different than what is typically done in contrastive learning, where augmentations are applied on the fly. Even when considering a fixed set of augmentations, the size of the graph would have to be many times the size of the original dataset, right? **Empirical Validation:** - Were the same positive samples (including label information) used when comparing to SimCLR? From the discussion in appendix D, I think they were not. If that is the case and the SimCLR experiment is purely self-supervised, why is there such a large difference between the known, novel and all classes? Shouldn't all the completely un-/self-supervised methods be agnostic to which class has labels (which they cannot use anyway)? - Using a ResNet-18 typically leads to about 90% linear probing and kNN accuracy on CIFAR10 for SimCLR. I appreciate that the evaluation is different and k-means is used to infer the clusters here. Nevertheless, I am surprised by the drastic drop in performance. Do the authors have an idea why this happens? Do the same SimCLR representations still achieve about 90% accuracy with linear probing / kNN classifier? - I appreciate that the paper is about the open-world setting, where there might be novel classes without any labels, so that one can only do clustering and not classification for evaluation. Still the label information is available for CIFAR10 and CIFAR100 and I would have found it interesting to see how the proposed method fares under the SimCLR evaluation protocol with a linear probe / kNN classifier. This setting is standard in the self-supervised learning world and thus makes the quality of the proposed method more interpretable to an audience more familiar with self-supervised rather than open-world representation learning. I assume SORL should outperform SimCLR since it has some label information available at train time, right? - Why is the performance for the "known" classes lower than that of the "novel" classes and why is the performance for "all" classes (typically) worse still than for the "novel" classes for the methods above the horizontal line in Table 1? For the open-world methods below the horizontal line, the performance is (typically) best on the "known" class, worse on the "novel" class and somewhere in between for "all" classes. This it what I would have intuitively expected. - The method LSD-C [1] achieves over 80% clustering accuracy on CIFAR-10 without using any labels (and no supervised training of a shallow classifier). It might therefore be a relevant competitor. **Minor:** - The batch size of 512 is rather small for contrastive learning. Why is that? - Are the different islands of points that share both color in shape in Fig 3b meaningful? Do they encode perhaps the size or position? **References:** [1] Rebuffi, Sylvestre-Alvise, et al. "Lsd-c: Linearly separable deep clusters." Proceedings of the IEEE/CVF international conference on computer vision. 2021.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

4 excellent

Limitations

Limitations were not explicitly discussed. I suggest a more detailed discussion of how their theory and assumptions hold up on real world data, see weaknesses above.

Reviewer 66eW7/10 · confidence 3/52023-07-04

Summary

This paper presents spectral open-world representation learning that aims to learn low-rank approximation of a constructed adjacency matrix. From this perspective, the authors study how the label information help (or hurt) the classification performance by analyzing the error bound. Experiments on image classification benchmarks show the effectiveness of the proposed approach.

Strengths

1. The formulation of open-world representation learning problem as a low-rank approximation of a weighted graph adjacency matrix is novel and reasonable. 2. The representation is clear. The provided illustrative example is helpful for understanding the theory. 3. The new ORL approach itself is simple yet effective.

Weaknesses

1. The theoretical analysis presented in the main text is only applicable to the proposed SORL approach. Generalizing to other approaches needs independent considerations as stated by the authors in section 6. 2. The empirical validation seems somewhat disconnected with the main theoretical result (theorem 4.1 and 4.2) as it only focuses on the performance of SORL. It would be better to conduct experiments to directly validate the main theorems e.g. by showing the connection between performance improvement by adding different label information and factors affecting $\Delta_{\pi_s}(\delta)$.

Questions

1. Is it possible to generalize the analysis to the SSL and supervised learning settings? 2. Is the spectral gap in theorem 4.1 crucial for the result?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

4 excellent

Contribution

3 good

Limitations

Please see the weakness section.

Reviewer 7sHp7/10 · confidence 3/52023-07-06

Summary

This paper formulate the open-world representation learning using the graph. This paper provides theoretical analyses for the framework (Thm 3.1) as well as the framework's performance (Thm 4.1 & 4.2). The experimental results show that the framework outperforms the exiting method in the open-world learning setting.

Strengths

Overall, I enjoy reading this paper and have a positive feeling. - Formulation. Of Eq. (6). By Thm. 3.1 this formulation corresponds to the SVD of normalized adjacency matrix Eq. (4). See more on the question (*). - The theoretical insights (Thm 4.1 & 4.2). The analyses using $k$-means measure strengthen the formulation. Particularly, the insight in L251-L258 is interesting; knowing more about green light increases the knowledge about the green apple, but not the flowers. This corresponds to our intuition.

Weaknesses

There's little I can say, but if I need to raise one, I would like to point out on the experiment. The experiment is on the only one split, but I am curious the other splits. Ideally it'd be nice to have results like Fig 4 in [79], y-axis accuracy vs x-axis # of the known labels. By seeing this, we can know "how many pictures of the green light do we need to learn in order to know the green apple?" I also would like to know the computational time, see more on the question section (**) From this viewpoint, I would give 6.

Questions

- How much do the things need to be different in order to learn? By learning something, how much the SORL can distinguish some similar things? If we do not know anything about the green apple, can we increase the chance to distinguish the different variety of the green apple by learning green signals? Maybe the difference between the variety of the green apple is embedded in the spectral gap, but do you have any insights on this? - I'm just curious how did you come up with Eq. (6)? - This is not the question but a comment connecting to (*) in the strength section. The Eq.(5) gives the SVD, but in this case Eq.(5) gives the eigendecomposition. Since the normalized Laplacian $L$ is defined as L := I -A, the Eq. (5) gives the k smallest eigenvalues. Thus, since the $k$ smallest eigendecompoition of the normalized Laplacian corresponds to the relaxed solution of the normalized graph cut, the SORL is actually corresponds to the relaxed solution of the normalized graph cut (see Sec.5.3 in [i]). From this viewpoint, as far as I understand, Thm 3.1 can be more powerful for the spectral clustering community -- the Eq. (6) corresponds the result of the spectral clustering (i.e., the relaxed solution of the normalized graph cut). With this, the proposed method is not surprising (in a good sense); the proposed SORL is an open-world learning hinted normalized graph cut. From this view, I'd like to connect the point (**), if this method is faster than SVD, this indicates that there can be some faster way of the spectral clustering? - This is also minor but the authors might want to cite [i], which is the one of the most cited paper in spectral clustering. [i] Von Luxburg, Ulrike. "A tutorial on spectral clustering." Statistics and computing 17 (2007): 395-416. ---- POST REBUTTAL I'm satisfied with the authors' rebuttal, therefore I increase my score from 6 to 7.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

N/A

Reviewer iP9L7/10 · confidence 3/52023-07-07

Summary

The paper tackles the domain of open-world representation learning, which aims to learn representations that can correctly cluster samples in the novel class and classify samples in the known classes by utilizing knowledge from the labeled data. Notably, the motivation of this paper is to provide a theoretical foundation in this area by introducing a novel graph-theoretic framework specifically designed for open-world settings. Theoretically, the crux of the framework involves characterizing clustering through graph factorization. A new algorithm called Spectral Open-world Representation Learning (SORL) is introduced, which essentially involves spectral decomposition on the graph. The authors provide theoretical guarantees by deriving a provable error bound on the clustering performance for both known and novel classes. Practically, the paper also offers empirical evidence, demonstrating that SORL is competitive with, or even superior to, existing methods on benchmark datasets.

Strengths

The paper is well-written with clear motivation and structure. Overall interesting problem; good mathematical exposition; solid theoretical results and analysis; interesting and good experimental results

Weaknesses

1. The experiments are not sufficient. Other large commonly used benchmark dataset is not used, like ImageNet-100. 2. The paper has a very good theoretical analysis. But it is still unclear what contributes to the performance improvement.

Questions

1. It would be interesting to see the results in a larger benchmark dataset, ImageNet 100. 2. What is the effect of using different known and novel classes ratio besides 50%? Like, 0.25, 0.1. 3. How do you choose the hyper-parameters, $\eta_{u}$, $\eta_{l}$? 4. In the supplementary material, it is mentioned that pre-training is used. I wonder why this pre-training is necessary for this method and what it will be without pertaining.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

4 excellent

Contribution

4 excellent

Limitations

The broader impact has been discussed. It has a positive potential impact on the community.

Reviewer 7sHp2023-08-10

Increasing my score from 6 to 7

Thank you very much for the rebuttal. I am satisfied with the additional experiments. I think that the experimental results make sense. I'd like to increase my score from 6 to 7. By camera-ready (you don't have to do in the rebuttal period in my opinion), I'm curious for the other split for the other methods, ideally one from open world based (e.g., ORCA) and one from non-open world (e.g., FixMatch), if this makes sense. Sometimes unsupervised is not possible, but I believe that probably you can use only one instance instead. The reason why I propose such experiment is that the SORL does not seem to grow fast in "novel," which is an important category for the open world learning setting as far as I understand. From 0% to 10% the accuracy seems to grow fast, but from 10 to 25% accuracy does not change, to 50% the accuracy increase seemingly marginally. In contrast, "seen" seems to grow, which makes sense. But we can only judge this by comparing with others. If you figure out that the SORL is "slow" for the novel by doing the additional experiments, do you have any insights for slowness from the theory perspective? Is 10% all you need for novel? I expect to have some answers for this by camera-ready. Even if your method is somewhat weaker than the other methods in some sense from this perspective, I am assured your method is promising since your method almost beats the others at 50%. I guess that the additional experiments will not reduce the value of this study and therefore I request this by the camera-ready, only if this makes sense. At the same time, I understand that the additional experiments for the other methods may not be possible since the other results may be brought from the other papers. Thank you very much for the other answers for my questions. All of these make sense. I enjoyed reading the rebuttal as well as the manuscript. --- EDIT: At this point I do not know how to update the score in my main rebuttal as edit button disappears. Thus the visible rating remains 6. But don't worry, my rating in my mind at this point (as of 10th Aug) is 7. I'll update as soon as I figure out how to do so.

Reviewer 1J6p2023-08-11

Thank you for your reply! **Discussion of the augmentation graph:** Your clarifications are helpful, especially the mention of generative augmentations and the limited relevance of the order of augmentation probabilities for the Theory in Sec 4. **Justification on the graph setup for the toy example:** - *Why only consider shape and color* Of course, a toy example deliberately has fewer properties than a real example. My confusion stemmed from a misunderstanding of the augmentations. I had not realised that all the augmentation probabilities are simply completely independent of the position and size of the shape, e.g., an object has the same probability to be augmented to any other position or size without changing shape and color, namely $\tau_1$. This should actually have been sufficiently clear from Fig 2a). So in a way, there is "perfect augmentation" to unlearn the irrelevance of size and position. - *Why assume a non-zero $\tau_c$?* I now understand that the order of the augmentation probabilies does not matter for correctness of the theory in Section 4. But the connection of unlabelled to labelled data, i.e., $\mathfrak{l}$ does enter Thm 4.1 and Thm 4.2. In that light, I appreciate the first part of the proposed limitation statement in the general comment. - *Why assume $\tau_c > \tau_s$?* Thanks for clarifying that the validity of the theory in Sec 4 does not hinge on the order of these probabilities. Do I understand correctly that choosing $\tau_c > \tau_s$ is a deliberately problematic setting, in which augmentations maintaining the category of interest (shapes) are less likely than augmentations that change it and instead maintain a nuisance property (color)? Without labels this augmentation setting would naturally cluster by color rather by shape, as depicted in Fig 3a) Adding the labels for shape cube rectifies the erroneous focus of the augmentations on the color. If one instead chose $\tau_c < \tau_s$, no fix would be required because the resulting clustering is automatically by shape (although perhaps there is still a marginal benefit from using labels for some shape). I think I had missed the first part of this, i.e., that there is anything to fix because the augmentations strengths do not align with the category of interest. From what I understand, a situation similar to $\tau_c > \tau_s$ is also the one in which Thm 4.2 offers the largest benefit: high connection to the labelled class and high inter class similarity (from large $\tau_c$) and low intra-class similarity (from low $\tau_s$). Thanks again for the clarification! **Empirical illustration of the main theorems:** Thank you for including these results in the additional pdf! Especially the main term in Thm 4.1. is impressively accurate. Thanks for clarifying in the proposed limitations section that one needs access to the full adjacency matrix to evaluate the terms in Thms 4.1, 4.2. Sorry if this is a trivial question, but I do not get the connection entry in Tab 2. From the top of page 13, I thought it would be $\mathfrak{l}_{sphere} - 1 / N = 0.5\tau_c -0.0001 = 0.1499$. What am I missing? **Justification on results of SimCLR:** Thank you for computing linear probing and kNN classifier accurarcies on your SimCLR representations. This helps dissecting which part of the performance drop is due to the evaluation protocol and which is due to the representations themselves. It seems that, indeed, the main difference in numbers is due to the evaluation setting. Nevertheless, the linear probing and kNN classifier accuracies are still lower than what as been reported for a ResNet18 on CIFAR10, e.g., in https://openreview.net/pdf?id=B8a1FcY0vi, Tab. 1. It will probably be difficult and not curcial to pinpoint exactly where the difference comes from. Just to confirm my understanding: The SimCLR representation learning is completely unsupervised and uses all of CIFAR10 (train+test set) during this unsupervised training, right? **Performance on LSD-C:** Many thanks for evaluating this additional method. I find it intruiging that it performs on par with RankStats, despite begin entirely unsupervised.

Authorsrebuttal2023-08-15

Thanks for your response!

Thank you for your invaluable feedback! We're pleased to note that our clarification has resonated, and your understanding of the augmentation graph and toy example aligns perfectly with our intent. Addressing your further queries: + *On the setting of $\tau_c > \tau_s$:* Yes! We choose the setting deliberately so that the clustering will be misled by the colors. It subsequently offers the space to improve by clustering samples on shapes, upon incorporating labels. Your interpretation from the view of Thm 4.2 also perfectly aligns with the intuition! + *On the value of $\mathfrak{l}_{sphere}$*: Good catch! At a high level, the discrepancy originates from the scaling factor. To elucidate, our theory hinges on the premise that T's row sums to 1. Given $1 = \tau_1 >> \tau_c > \tau_s$ (footnote on page 5), we can follow the equation in the Appendix (top of page 13), which is simplified and has no normalization. We can fix the glitches by setting $\tau_1 = 0.6$, $\tau_c = 0.3$, $\tau_s=0.1$, the value of $\mathfrak{l}\_{sphere}$ now aligns with the reviewer's proposition of 0.15. In this case, concerning the estimation gap, both $\Delta_{kms}(\delta)$ and the estimation now become 0.0027 (when $\delta=0.01$). For the broader audience, we will make the simulation code open-sourced together with the SORL implementation upon the paper's publication. + *On SimCLR's training setup*: The discrepancy in the SimCLR's performance requires a deeper examination of the different training settings. For the reviewer's reference, we directly use the training script at https://github.com/HobbitLong/SupContrast. Regarding the question, the answer is **yes** for the purely unsupervised setting but **no** for the `train+test` split. Specifically, we perform the unsupervised training on the `train` split but report KNN/LP performance on the `test` split. Thank you again for the comments and suggestions to help us improve the manuscript!

Reviewer 1J6p2023-08-15

Thank you for the additional information! I maintain my score.

Reviewer 66eW2023-08-18

Thank you for your response. Most of my concerns are addressed. I particularly feel empirical validation of the theory (beyond evaluation of the proposed method) is valuable and encourage the authors to consider incorporating relevant discussions in the paper. I increase soundness to 4.

Reviewer iP9L2023-08-20

Thanks for your responses. Having read the rebuttal, I will increase my initial score.

Authorsrebuttal2023-08-20

Thank you!

We would like to thank the reviewer for taking the time to read our rebuttal and for your positive feedback! Best, Authors

Program Chairsdecision2023-09-21

Decision

Accept (spotlight)

© 2026 NYSGPT2525 LLC