Latent graph inference (LGI) aims to jointly learn the underlying graph structure and node representations from data features. However, existing LGI methods commonly suffer from the issue of supervision starvation, where massive edge weights are learned without semantic supervision and do not contribute to the training loss. Consequently, these supervision-starved weights, which may determine the predictions of testing samples, cannot be semantically optimal, resulting in poor generalization. In this paper, we observe that this issue is actually caused by the graph sparsification operation, which severely destroys the important connections established between pivotal nodes and labeled ones. To address this, we propose to restore the corrupted affinities and replenish the missed supervision for better LGI. The key challenge then lies in identifying the critical nodes and recovering the corrupted affinities. We begin by defining the pivotal nodes as $k$-hop starved nodes, which can be identified based on a given adjacency matrix. Considering the high computational burden, we further present a more efficient alternative inspired by CUR matrix decomposition. Subsequently, we eliminate the starved nodes by reconstructing the destroyed connections. Extensive experiments on representative benchmarks demonstrate that reducing the starved nodes consistently improves the performance of state-of-the-art LGI methods, especially under extremely limited supervision (6.12% improvement on Pubmed with a labeling rate of only 0.3%).
Paper
Similar papers
Peer review
Summary
The authors figure out that the graph sparsification operation results in the supervision starvation problem in latent graph inference (LGI). They propose to identify k-hop starved nodes and diminish the starved nodes by incorporating a regularization adjacency matrix into the initial one. They further reduce the computational cost by using CUR matrix decomposition and tackle the weight contribution rate decay problem via some simple strategies. The effectiveness of the proposed method is validated on well-known benchmarks. Update after rebuttal: The authors have addressed my concern through the rebuttal. The score remains unchanged.
Strengths
1. This paper is well-written and easy to follow. 2. This paper identifies the supervision starvation problem for LGI. 3. The proposed method is well-motivated and theoretically justified. 4. The proposed method is model-agnostic and can be seamlessly integrated into various LGI models. 5. The empirical results are significant on benchmarks.
Weaknesses
From the analysis in Section 4.2, the proposed method may be sensitive to the hyper-paramters $\tau$ and $\alpha$.
Questions
In Eq. (4), the refined matrix $\widetilde{\mathbf{A}}$ becomes a weighted adjacency matrix when the positive parameter $\alpha \neq 1$. Will tuning $\alpha$ help with methods developed for unweighted graphs?
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
4 excellent
Presentation
4 excellent
Contribution
4 excellent
Limitations
N/A
Summary
The paper points common LGI methods suffer from the issue of `supervision starvation`. It also observes this issue is actually caused by the graph sparsification operation. To address this problem, the paper proposes to restore the corrupted affinities and replenish the missed supervision. It presents the `CUR matrix decomposition` to reduce the computational burden and eliminates the starved nodes by reconstructing the destroyed connections The method is model-agnostic and can be seamlessly integrated into existing LGI methods. Extensive experiments show promising results.
Strengths
1) Originality. The identifies graph sparsification is the main cause of the supervision starvation (SS) in LGI. The paper is solid in analyzing problems with insights. 2) Quality. This method is supported by both theoretical (Theorem 1, 2) and experimental aspects, having strong persuasiveness. 3) Clarity. The paper is well written and organized. 4) Significance. LGI is a common task graph learning. This method can be potentially applied in many situations.
Weaknesses
1) The paper should clearly explain why using `CUR Decomposition` is more efficient. The experimental results only show a higher accuracy, but does not reflect the efficiency. So this advantage is not well supported. 2) The section3.2 and 3.3 are not easy to follow. The reviewers and potential readers may prefer a popular version.
Questions
1) The $L_{reg}$ should have a clear mathematical form in paper in Equal 9. 2) The authors are encouraged to discuss more about SoTA GNNs [15,23,37,40] since the proposed approach is model-agnostic. The reviewer is curious why these methods were mentioned in Section1 but not compared in experiments.
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
3 good
Presentation
3 good
Contribution
3 good
Limitations
The LGI is a common task in graph learning and all experiments are conducted on public and popular datasets. The reviewer see no negative societal impact and limitations of this work.
Summary
The paper proposes a method for latent graph inference (aka graph structure learning) based on the idea of mitigating the supervision starvation problem present when jointly learning the underlying graph structure and node representations. The paper claims that the supervision starvation problem is caused by the sparsifier operation present in many existing models in the literature. It further proposes to recover (some) starved edges to resolve the issue and shows that reducing the number of starved edges consistently improves the performance on multiple benchmarks.
Strengths
The problem of latent graph inference and the supervision starvation problem present in many existing approaches are important directions that are recently gaining more and more attention. The paper proposes an approach to identify the starved edges and add them back to the graph structure that is backed up by CUR decomposition and theoretical results. The final analysis shows the proposed strategy improves the performance of state-of-the-art LGI methods. The method is shown to be more effective when supervision is less.
Weaknesses
W1: One main weakness of the model is that the results are obtained using the average of the top five best testing performances. I understand that this setup was fixed across multiple baselines and datasets but to measure the generalization of the model, I would expect the results to be an average over testing performance of the models corresponding to the best validation performance. This is especially important as the charts in Figure 2 show a lot of instabilities in the test loss of multiple models. W2: the proposed model adds some edges to the graph structure and compares itself to the base model without those edges. I suggest adding a baseline as the base model + random edges to make sure the performance obtained is not just because there are more edges present.
Questions
Q1: Can you elaborate more on what models M_U and M_R are referring to? My understanding so far is that M_U is referring to Equation 6 and M_R is referring to Equation 8. Q2: Can you explain what edge weights have been used on both existing and starved edges? Is this the real-valued edge from the output of the graph generator for both type of edges? Q3: What type of graph generators has been used to obtain the results in Table 1?
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
Soundness
3 good
Presentation
3 good
Contribution
3 good
Limitations
One limitation of the model is that Theorem 2 only works for k of 1 and 2. For many academic datasets, using one or two layers of GNNs is enough. However, in many real-world applications, deeper GNNs are needed to send the long-range dependencies. The paper does not provide any insights on how the idea can be extended for higher values of ks.
Summary
This paper proposes a new method for the latent graph inference problem. The motivation of the new method is the existence of supervision starvation nodes caused by graph sparsification operation. To reduce the number of supervision starvation nodes, the authors propose a CUR matrix decomposition based method to add an additional adjacency matrix to the original sparse adjacency matrix with some constraints. The extensive experimental results show that the proposed idea is effective on various datasets and different base models.
Strengths
1. The paper is well written. It is easy to follow the idea and understand the motivation and methodology design. 2. Although the method is simple, it is effective and with sufficient analysis. 3. The performance looks promising.
Weaknesses
The reason why some nodes may receive no supervision is also because of the only one-layer GNN. If there are multiple layers of GNNs, the supervision information will propagate in the graph to achieve semi-supervised training, which is the core of most of GNNs. If there are multiple layers of GNNs, is that still reasonable to define ‘supervision starvation’? In Line 155, why is deeper GNN not an optimal solution to reduce the starving nodes? From the Figure 1, we can see that increasing GNN to 4 layers could reduce the starving nodes from near 3,000 to only near 500. Since matrix A is dynamically generated from a latent graph generator, will the U or Q dynamically change as well? Are they dependent on generated A? If so, is that differentiable?
Questions
see above
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
3 good
Presentation
3 good
Contribution
3 good
Limitations
Yes
Summary
The paper introduces a model-agnostic enhancement for current LGI (latent graph inference) methods, aiming to address the assumed issue of unlearned features in a substantial number of nodes and edges, which are believed to negatively impact LGL's generalization performance. It proposes to learn a weighted residual graph, focusing on unlabeled nodes without labeled neighbors and intending to link them with some labeled nodes. Additionally, the proposed method applies a sparsity constraint to the residual graph, preventing significant alterations to the underlying graph topology learned from the base LGI model. Finally, the paper evaluates the effectiveness of this approach by integrating it with several prior LGI models, and finds modest improvements in standard graph learning tasks.
Strengths
The paper's strength lies in its investigation of the important problem of optimizing the structure for neighborhood aggregation in graph neural networks. It introduces a method that addresses the issue of "supervision starvation" by adding labeled neighbors to initially supervision-starved nodes, which is demonstrated to bring benefit to existing LGI methods, especially on larger graphs.
Weaknesses
1. The paper's proposed method shows moderate improvements, with limited statistical significance on small graphs (Cora and Citeseer). Larger graphs display more significant enhancements. The relationship between improvement and graph size should be explored for better insights. Further improvements for small graph scenarios are needed. 2. The paper lacks clarity in explaining the generation of the residual adjacency matrix. The emphasis on the relationship with CUR decomposition should be weaken since the learning criteria for weights in matrix $\tilde{U}$ are unrelated to reconstructing the target matrix $Q$. Furthermore, the method of selecting "supplementary adjacent points" from the labeled nodes for each 1-hop starved node remains unclear. The authors should be aware that "randomly selecting a subset of $\tau$ nodes from $c$ labeled nodes" is a different statistical event compared to "picking each labeled node at a probability of $\frac{\tau}{c}$". Improved explanations of these aspects are necessary to avoid confusion and strengthen the paper's presentation. 3. The paper lacks significant theoretical and technical novelty. Certain modeling choices are presented without adequate theoretical justification to demonstrate their importance or sensitivity analysis to show their insignificance. Specifically, the content in the paper should be enriched or reorganized to address the following research questions: (1) the reasons and mechanisms behind the significance of k-hop starved nodes in training a GNN through semi-supervised learning (i.e., why they are pivotal nodes), and (2) a comprehensive evaluation of the advantages and disadvantages of selecting k=1 to define the residual bipartite graph between k-hop starved nodes and labeled nodes, emphasizing how the benefits outweigh any drawbacks. Addressing these questions would enhance the paper's contributions and strengthen its overall impact in the field. Given the aforementioned issues, a major revision of the current version of the paper is recommended before acceptance.
Questions
1. Could the authors provide further elaboration on why the k-hop starved edges are defined as "having at least one endpoint being a $k$-hop starved node"? If the main objective is to identify edges whose weights are not trained in a $k$-layer GNN, wouldn't a more appropriate definition be "both endpoints are $(k-1)$-hop starved nodes"? (0-hop starved = unlabeled) 2. Could the authors provide an analysis of why adding connections between 1-hop starved nodes and labeled nodes improves the model's generalization? It appears that while introducing supervision signals to some unlabeled nodes, these additional connections to the labeled nodes may make them less "similar" to the unlabeled nodes. For example, the degree distribution of labeled and unlabeled nodes might be similar in the original dataset, but after the structural modifications proposed in the paper, the labeled nodes may exhibit significantly higher degrees than the unlabeled nodes. Wouldn't such divergence in the distribution of labeled and unlabeled nodes negatively impact the model's generalization? An explanation of these potential effects would help clarify the benefits and implications of the proposed structural modifications.
Rating
5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
3 good
Presentation
2 fair
Contribution
3 good
Limitations
N/A
Thank you!
Thank you very much for your quick response and valuable comments. We will try our best to revise our manuscript accordingly. Aagin, thank you so much for helping us improve the paper!
Sincerely Expecting Further Discussion
Dear Reviewer Qcor, It is only 4 days away from the end (8/21) of the discussion period, and we haven't received any feedback from you regarding our responses. We are here to see if you could spend a few minutes checking our responses. Your concerns mainly relate to graph size, the generation of residual graphs, the definition of starved nodes, novelty, and generalization, for which we have dedicatedly provided more analysis and explanations to clarify our method. We do want to hear your further opinion about this, which is essential for us to improve the work. Thank you very much! Best regards, Authors
Thank you for the rebuttal. Here are some clarifications of my questions.
I thank the authors for providing a thorough explanation in response to my concerns. Following a careful review of the authors' rebuttal, I recognize the paper's importance in tackling the "supervision starvation" issue, which presents a novel research problem worthy of investigation. **Hence I decide to raise the score from 3 to 4**, and would like to keep updating the score if following questions can be discussed in revision: 1. About model generalization. The concept of "model generalization" usually pertains to how well a machine learning system performs on data it hasn't encountered before. However, in the context of transductive learning, there might be uncertainty regarding whether the test set can be truly considered "unseen" during the training phase. Despite this, my intention was to highlight to the authors that introducing connections between labeled and unlabeled nodes can lead to a substantial rise in the degrees of labeled nodes. This outcome might not always be advantageous, as it could cause the degree distribution of labeled nodes to differ from that of unlabeled nodes. My main skepticism revolves around the question of whether altering the distribution of node degrees will have an adverse effect on the model and to what extent. For instance, in the context of a node classification problem in a citation network, the degree of a node might be associated with the impact of the corresponding article. If there is a predictive connection between an article's impact and its subject label, the relationship between the degree and the label that the model learns from labeled nodes might not apply well to unlabeled nodes. This divergence in their degree distributions could potentially hinder the model's performance on unlabeled data. 2. About Graph size. While I concur with the authors regarding the non-influence of graph size on model performance, it's notable from Table 1 that the significance of performance enhancements is notably higher for PubMed and ogbn-arxiv compared to Cora and Citeseer. The distinction in benchmark groups includes graph size. While it's acknowledged that other potential variables are uncontrolled, we can't definitively conclude from Table 1 that graph size impacts model performance. I anticipate insights on "situations where the model excels" given that some improvements listed in Table 1 are less significant than others. 3. About the definition of $k$-hop starved weights. In lines125, 126, the paper writes: > According to the definition presented, it is evident that k-hop starved weights are precisely the ones that receive no semantic supervision from labels My understanding is that "$k$-hop starved weights" corresponds to "untrained weights in a $k$-layer GNN." To illustrate, consider a graph with 4 nodes: $u_1$, $u_2$, $u_3$, and $u_4$. The edges are $(u_1, u_2)$, $(u_2, u_3)$, and $(u_3, u_4)$. If nodes $u_1$ and $u_4$ are labeled, for a 1-layer GNN, the untrained weight is on edge $(u_2, u_3)$, hence $(u_2, u_3)$ is a 1-starved edge. However, according to Definition 2, none of the four nodes are considered 1-starved nodes. This situation appears perplexing, could the authors please provide clarification? 4. About the confusion in the selection method of sampling $\tau$ nodes from $c$ labeled nodes. The authors are correct, no revision is needed for this part. 5. About generation of the residual matrix. I recognize that the intention is not to generate a matrix for reconstruction purposes. As a result, I recommend considering a reduction in the association between the proposed method and CUR decomposition, which is primarily designed for matrix reconstruction. I personally don't find it necessary to introduce CUR decomposition to interpret the U matrix.
Thank you very much for raising the score!
Dear Reviewer Qcor, **Thank you so much for your reply and raising the score! We are very pleased that you recognize our paper’s importance**. After reading the clarifications carefully, we find that your questions are important and worth further exploring, which will improve our paper to a higher level. We address your questions below. Q1. Thanks for your further clarifications on this question. We concur with you that introducing connections between labeled and unlabeled nodes can lead to a substantial rise in the degrees of labeled nodes. It is actually a good question whether altering the distribution of node degrees will have an adverse effect on the model. To be honest, we have not yet observed any adverse effects from the current experimental results. We can provide a basic insight as to why introducing connections between labeled and unlabeled nodes helps the models. As we know, an unlabeled node must belong to a specific class, and in the node classification task, a specific class must have some labeled nodes. As a result, we can derive the prior assumption that a good classification model will make an unlabeled node closer to some labeled nodes of the same class. That’s why we select the closest labeled nodes as the supplementary adjacent points for the starved nodes. If we compulsorily add such connections between the unlabeled node and its closest labeled nodes, the prior information will be exploited during training, helping the models work better. Nevertheless, it is still worth exploring whether the divergence in degree distributions will potentially hinder the model’s performance. What the exact relationship is between the degree distributions and the model’s performance is a difficult and open question that deserves in-depth exploration. We think the reviewer provides us with a good potential direction to research deeper. Due to time limitations, we will explore this in the future. Q2. We are pleased that you agree with us regarding the non-influence of graph size on model performance. For the situations where our model excels, we provide some basic insights as follows. First, we think the labeling rate of nodes is the key factor that affects the significance of improvement. As we can see from Table 1, PubMed has the lowest labeling rate, while the improvement is the most significant. Second, the improvement also depends on the baselines we used. Different baselines have different graph inference strategies and graph sparsification processes, which also makes the improvement significance different. For example, we can observe that our methods obtain more than 1% improvement for the GCN&KNN model on the Cora and CiteSeer datasets, while the improvement for the GCN+KNN model is not significant. In summary, several potential factors (e.g., labeling rate, graph sparsification process, datasets, baselines, etc.) may affect the improvement of our models. We only provide some basic insights on this since it is actually difficult to say exactly which situation our method is definitely good in. Q3. Thanks for pointing this out and providing an illustration. We have carefully read your clarifications, and we finally recognize that your understanding is correct. Note that a k-hop starved node also qualifies as a (k-1)-hop starved node. It is actually more appropriate to define the k-hop starved edges as “both endpoints are (k-1)-hop starved nodes (0-hop starved = unlabeled)”. We will update the definition accordingly in the revision. Thanks again for your kind reminder and suggestion! Q4. We are very pleased that our responses can successfully address your concern on this point. Thanks for the time spent reading the detailed responses! Q5. Due to space limitations, we had to delete some explanations on the target matrix Q at the first rebuttal. Now, let us discuss this more. In fact, our initial intention was to construct the target matrix Q (in Eq. 5) and use it as the regularization matrix B (in Eq. 4). It is, of course, sensible and feasible. However, we found a potential drawback to this design. That is, the reconstruction of target Q requires matrix multiplications of three matrices, which is very time-consuming. To tackle this problem, we unexpectedly found that only constructing matrix U is enough to solve the supervision starvation problem. That’s the reason why we emphasize and focus on matrix U in the manuscript. We recognize that your suggestion is also reasonable. To make a better association between the proposed method and CUR decomposition, we will discuss the above initial design in the revision, add a variant that uses the target Q as the regularization, and provide more experimental results. In comparison with reduction, we think this is a better way to make our paper comprehensive and consistent with a better logical flow. **We will discuss these questions in the final revision. We sincerely thank the reviewer for helping us improve the paper!** Best regards, Authors
Follow-ups
1. I am very interested in the response to Q5 in the above comment. Could the authors confirm for me that there are two variants of $\Gamma(\tilde{\mathbf{U}}, n)$, one is padding 0 into $\tilde{\mathbf{U}}$ as introduced in the paper, another is $\Gamma(\tilde{\mathbf{U}}, n) = \mathbf{C}\tilde{\mathbf{U}}\mathbf{R}$, and the first variant is more efficient than the second variant? 2. I'm pleased that I grasped the notion of $k$-starved weights accurately. I recommend to the authors that they consider adding a supplementary note clarifying that self-connections aren't included within the definition of any neighborhood. Such clarification would prevent potential confusion concerning nodes classified as $0$-starved and $1$-starved. 3. As a follow-up on the discussions about model generalization, I find the authors' explanations to be persuasive. Within the context of transductive learning, the discrepancy between labeled and unlabeled nodes might carry less weight compared to the challenge of supervision scarcity. Nonetheless, this aspect could present a possible concern for inductive learning scenarios. While I comprehend that delving into this matter goes beyond the paper's current scope, it would certainly be valuable to include it in future discussions.
Thank you for your reply and further questions!
Dear Reviewer Qcor, Thank you very much for your reply and further questions! We address your concerns below. Q1. Yes, you are right. There are two variants, and the first variant (padding 0) is more efficient than the second variant. We can also provide experimental results to demonstrate this point. Due to time limitations, we will add the results to the final revision. Q2. Thank you very much for this suggestion. It is sensible to make such a clarification. We will add a supplementary note to clarify that self-connections aren’t included within the definition of any neighborhood. Thanks again for pointing this important information out for us! Q3. Thanks for your agreement with our explanations. We totally concur with the reviewer that it is valuable to include this matter in future discussions. We thank the reviewer for this important suggestion, and we will definitely discuss this point in our final revision. **Thanks again for providing us with so many important suggestions and new insights.** Best regards, Authors
Upscore decision
The paper's notable strength lies in its ability to demonstrate to the community that tackling the issue of supervision scarcity holds substantial promise as a direction for advancing latent graph learning methodologies. While certain concerns related to presentation clarity were initially raised, the authors have committed to making necessary enhancements, which have been duly addressed. Personally, I believe that encouraging papers that introduce intriguing topics to the community is essential, and thus, I am biased towards recommending the acceptance of this paper. Wishing the authors the best of luck.
Thank you very much for recommending the acceptance of our paper!
Dear Reviewer Qcor, **Thank you very much for recommending the acceptance of our paper**. We are very pleased that you can recognize our contributions and provide us with valuable suggestions and new insights! We will try our best to revise the manuscript accordingly. Again, many thanks for helping us improve the paper! Best regards, Authors
Additional Comments
Thanks for your response. I still believe that these SoTA methods should be compared, at least one of them, instead of just talking. At least an experiment result of them should be put on to show their shortcomings, Otherwise, I would think the experiment is insufficient.
Thanks for your reply!
Dear Reviewer se4r, Thank you very much for your reply and instructive suggestions. We are doing the experiments you requested now. We will try our best to show the experimental results before the discussion period. Thanks again for your help in improving our paper! Best regards, Authors
New experimental results.
Thank you again for your constructive suggestions. We add one more SOTA method, Pro-GNN [15], for experiments. We use their source codes and adopt the same data partitioning. In fact, Pro-GNN does not infer a latent graph from scratch based on the node features. It requires a prior semantic graph as the input. To align with our latent graph inference task, we construct a KNN graph as the prior graph for Pro-GNN. The experimental results on the Cora and Citeseer datasets are shown below (it cannot be trained on other datasets directly due to the out-of-memory issue). As we can see, our proposed methods still achieve better performance. | Datasets | Cora | Citeseer | | --- | --- | --- | | Pro-GNN | 70.73 ± 0.35 | 71.76 ± 0.14 | | Pro-GNN_U (ours) | 70.76 ± 0.32 | 71.86 ± 0.95 | | Pro-GNN_R (ours) | 70.85 ± 0.39 | 72.18 ± 0.33 | We hope this can address your concern. And again, thanks for helping us improve the quality of our paper.
Thanks for authors reply!
Thanks for the authors' reply! I am still unsure about the reasonability of "supervision starvation". Can we call all the semi-supervised learning tasks 'supervision starvation' because only a limited number of data points have correct label information? I will keep my score as before.
Thanks for your reply and question!
Dear Reviewer Qy2b, **Thank you very much for your reply and keeping a good score for us!** *No, we cannot call all the semi-supervised learning tasks “supervision starvation”*. The “supervision starvation” problem exists in semi-supervised latent graph inference tasks, not general semi-supervised tasks. Let us explain in detail. As we know, there are two types of weights in latent graph inference methods: **network weights** (i.e., network parameters in GNNs) and **edge weights** (the values of the adjacency matrix). *When we say “supervision starvation”, we mean that some edge weights (not network weights) receive no supervision from any labeled nodes through the semi-supervised training loss*. This will lead to poor generalization since **these under-trained edge weights are inevitably used to make predictions for testing samples at the test time**. In fact, existing latent graph inference methods commonly suffer from this problem. However, for general semi-supervised learning tasks (latent graph inference is not required), there is only one type of weight: **network weights** (i.e., network parameters). In this case, *all network weights can receive useful supervised information from a limited number of labeled data points through the semi-supervised training loss*. **And the predictions of training and testing samples are calculated based on the same network weights, which are well-trained**. Apparently, the “supervision starvation” problem does not exist here. In summary, *“**supervision starvation**” refers to some **edge weights** receiving no supervision in latent graph inference methods*. Therefore, we cannot call all the semi-supervised learning tasks “supervision starvation”. We hope this can address your concern. Thanks again for your help in improving our paper! Best regards, Authors
Reviewer Response
Thanks for providing the new result and insights. Most of my concerns are now resolved and I have raised my soundness score and the overall rating accordingly.
Thank you very much for raising the score!
Dear Reviewer tzHX, We are very pleased that our responses resolved your concerns, and **thank you very much for raising the score!** We will try our best to revise our manuscript accordingly. Again, thank you so much for helping us improve the paper! Best regards, Authors
Decision
Accept (poster)