Summary
Overall, this paper makes the first attempt to theoretically revisit previous SHGL methods from the spectral clustering perspective in a unified manner. Specifically, this paper revisits SHGL from the spectral clustering and introducing a novel framework enhanced by rank and dual consistency constraints. Specifically, the proposed framework incorporates a rank-constrained spectral clustering method that refines the affinity matrix to exclude noise effectively. Additionally, the proposed method integrates node-level and cluster-level consistency constraints that concurrently capture invariant and clustering information to facilitate learning in downstream tasks.
Strengths
1. I really appreciate the idea that revisiting previous SHGL methos from the perspective of spectral clustering, which is interesting and may inspire some researchers in the graph learning as well as the self-supervised learning communities.
2. Theoretical analysis verifies the effectiveness of the proposed method, which divides the learned representations into distinct partitions based on the number of classes. Furthermore, the proposed method exhibits enhanced generalization ability than previous SHGL methods.
3. Extensive experiments on both heterogeneous and homogeneous graphs demonstrate the effectiveness of the proposed method. Visualization and case studies further verify the claims in this paper.
4. The proposed rank-constrained spectral clustering is novel and interesting, and it seems like can also be used to the graph structure learning in the homogeneous graph.
Weaknesses
1. In the Introduction, the authors claim that previous methods conduct message-passing relying on meta-path-based graphs and adaptive graph structures, which inevitably include noise. Are there any real examples to better illustrate the noise in meta-path-path based graph as well as the adaptive graph structures?
2. In the dual consistency constraints, the proposed method designs the node-level consistency constraint to capture the invariant information between the node representations and the heterogeneous representations. How about replacing the node-level consistency constraint with other common loss, such as InfoNCE [1]?
[1] Aaron van den Oord et al., Representation learning with contrastive predictive coding.
3. The authors design the rank-constrained spectral clustering to learn the affinity matrix for nodes belonging to the same node type. Although the visualization verifies the effectiveness of the affinity matrix, it would be better for the authors to add some ablations studies to further verify it, such as replacing the affinity matrix with a self-attention mechanism or simple cosine similarity.
4. What are the specific processes for different downstream tasks (e.g., node classification and node clustering)? It seems like training the proposed method first, and then freezing parameters of the model and applying outputted representations for downstream tasks.
5. The proposed method is designed for the heterogeneous graph while it can be implemented on both the homogeneous graph datasets and heterogeneous graph datasets. Is it just to replace the heterogeneous graph encoder with the graph convolutional layer? How to generate two different views for the dual consistency constraints?
6. It would be better to add some recent works about the self-supervised heterogeneous graph learning in the related work, especially those published in the past two years.
7. The paper needs further proofreading. For example,
- In Eq. (17) I know $q_i$ is the i-th projected representation. What’s the $\hat{\mathbf{q}}_{\mathbf{y}_i}$ actually means?
- The definitions of some symbols need to be further determined, such as \mathbf{I} in Definition 2.1. I guess it might represent the identity matrix.
- Some mistakes in Table 3, the Freebase dataset should be removed from the Table.
- The caption of Table 7 should be fixed.
Limitations
Yes, the authors have discussed the limitations and potential negative societal impact.