Certifiably Robust Graph Contrastive Learning

Graph Contrastive Learning (GCL) has emerged as a popular unsupervised graph representation learning method. However, it has been shown that GCL is vulnerable to adversarial attacks on both the graph structure and node attributes. Although empirical approaches have been proposed to enhance the robustness of GCL, the certifiable robustness of GCL is still remain unexplored. In this paper, we develop the first certifiably robust framework in GCL. Specifically, we first propose a unified criteria to evaluate and certify the robustness of GCL. We then introduce a novel technique, RES (Randomized Edgedrop Smoothing), to ensure certifiable robustness for any GCL model, and this certified robustness can be provably preserved in downstream tasks. Furthermore, an effective training method is proposed for robust GCL. Extensive experiments on real-world datasets demonstrate the effectiveness of our proposed method in providing effective certifiable robustness and enhancing the robustness of any GCL model. The source code of RES is available at https://github.com/ventr1c/RES-GCL.

Paper

References (65)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer A65y4/10 · confidence 3/52023-07-03

Summary

Inspired by the certifiable robustness of graph contrastive learning (GCL) is still remain unexplored, the authors develop the first certifiably robust framework in GCL by proposing a unified criteria to evaluate and certify the robustness of GCL. Specifically, the authors introduce a novel technique, RES (Randomized Edgedrop Smoothing), to ensure certifiable robustness for any GCL model, and this certified robustness can be provably preserved in downstream tasks. Experiments on 7 real-world datasets show that the proposed RES-GRACE achieves state-of-the-art performance.

Strengths

1. The presentation of the paper is very clear and the figures are easily digestible. 2. The numerical experimental results and visualization support the effectiveness of the proposed method.

Weaknesses

1. In this paper, the authors conduct experiments for both node and graph classification tasks over regular and large-scale graphs. It is good. However, the choice of baselines seems somewhat insufficient. Can the authors compare with some recent GCL baselines? E.g., RGCL [1] and GLCC [2]. 2. Can the authors provide some explanations why the performance of RES-backbone (e.g., RES-GRACE, RES-DGI) framework sometime is worse than the backbone on raw graph (however, their performance is always better than backbone under perturbation scenarios)? 3. From a clarity perspective, this work is, in my opinion, weakly motivated. It is not clear why the authors consider and start edgedrop smoothing to enhance the robustness of a GCL model. [1] Li, Sihang, et al. "Let invariant rationale discovery inspire graph contrastive learning." International conference on machine learning. PMLR, 2022. [2] Ju, Wei, et al. "Glcc: A general framework for graph-level clustering." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 37. No. 4. 2023.

Questions

Please see comments and questions in Weaknesses.

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

4 excellent

Contribution

2 fair

Limitations

The topic about certifiable robustness is quite interesting and is of high practical value. I believe this work has some potential things need to be addressed such as a better motivation and more details of comparisons in the experiments.

Reviewer vCmM5/10 · confidence 3/52023-07-06

Summary

This paper introduces a certifiably robust graph contrastive learning method called RES (randomized Edgedrop Smoothing). This paper (1) first introduces the criteria of how to evaluate and certify robustness, (2) and then introduces RES to ensure certifiable robustness, (3) finally, proves that the certified robustness can be transferred to downstream tasks.

Strengths

1. This paper is well motivated that current robust graph contrastive learning methods are not able to certify their robustness. 2. The theorem introduced in this paper looks interesting. 3. The experimental results on several benchmark datasets show that the proposed method could outperform baselines.

Weaknesses

1. The motivation of using $\mathbf{v}^+$ rather than $\mathbf{v}$ in Eq. (3) is not quite clear. Specifically, in Eq. (3), why do you use $s(h(\mathbf{v}'), h(\mathbf{v}^+)) > s(h(\mathbf{v}'), h(\mathbf{v}^-))$ rather than $s(h(\mathbf{v}'), h(\mathbf{v})) > s(h(\mathbf{v}'), h(\mathbf{v}^-))$. 2. $v$ and $v^+$, and the term "positive sample" are a kind of confusing in section 4.1 (after line 194). I think the "positive sample" after line 194 has a different meaning of the "positive sample" in contrastive learning. It is highly suggested that the authors find a better way to present the idea here. 3. The claim in line 225-226 that "the majority of them will possess identical structural vectors, that is, $\mathbf{v}\oplus\mathbf{\epsilon}=\mathbf{v}'\oplus\mathbf{\epsilon}$" seems too strong. If this is the real case, can you show the statistics in some real datasets? 4. Random dropping edges is one of the basic graph augmentation techniques that have been introduced in GraphCL [1] and GraphMAE [2]. What's the difference between RES and random edge drop in GraphCL and GraphMAE? [1] You, Yuning, et al. "Graph contrastive learning with augmentations." Advances in neural information processing systems 33 (2020): 5812-5823. [2] Hou, Zhenyu, et al. "Graphmae: Self-supervised masked graph autoencoders." Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2022.

Questions

1. In problem 1, what are $y$ and $y^\ast$? 2. What does "maximum of {$\mathbf{v}_1,\dots, \mathbf{v}_N$}" mean in Lemma 1? Can you show a concrete example? 3. There are many ways to define the probability shown in Eq. (5). For example, empirically, Gaussian is the most common and effective way. What are the advantages of your definition over a simple Gaussian?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

2 fair

Limitations

N/A

Reviewer F8a75/10 · confidence 3/52023-07-06

Summary

This study represents the first exploration of certifiable robustness in Graph Contrastive Learning (GCL). The authors propose a unified definition of robustness in GCL, addressing the existing ambiguity in quantifying its resilience to perturbations. The introduction of the Randomized Edgedrop Smoothing method is an interesting approach that applies randomized edgedrop noise into graphs to offer certifiable robustness on unlabeled data while minimizing the insertion of unnecessary edges. Theoretical analyses provided affirm the robust performance of their encoder in subsequent tasks. Extensive empirical experiments on various real-world datasets have been conducted, suggesting that the proposed method enhances the robustness of GCL models and provides certifiable robustness.

Strengths

1. The authors explore a novel area of certifying the robustness of Graph Convolutional Learning (GCL) against various perturbations. They introduce a comprehensive definition for robustness in GCL and design an innovative framework, Randomized Edgedrop Smoothing (RES), to validate this robustness. 2. A theoretical analysis provided in the study affirms that the representations learnt by their robust encoder can deliver a provable robust performance in subsequent tasks. 3. The authors have developed an effective training strategy for robust GCL, integrating the application of randomized edgedrop noise. 4. Comprehensive empirical evaluations demonstrate that their method can deliver certifiable robustness for downstream tasks.

Weaknesses

1. The selection of adversarial attack strategies in this study seems somewhat underwhelming given the marginal difference between clean accuracy and attack accuracy. It would be beneficial to employ more aggressive attack methods to genuinely assess the robustness of your proposed framework. 2. From what is shown in Table 3, the RES method appears to result in only slight improvements. Providing results from a broader range of graoh attack methods might lend more robust evidence to support your claims. 3. The paper seems to be missing a comparative analysis between the proposed Randomized Edgedrop Smoothing method and existing edge drop augmentation techniques in Graph Contrastive Learning (GCL). Including such an analysis could provide readers with a more concrete understanding of the advantages and potential improvements brought about by the RES method.

Questions

1. Could you clarify the operation of the evasion attack in the context of transductive node classification within Graph Contrastive Learning (GCL)? In a scenario where structural perturbations are incorporated into the test set, however, it's worth noting that the GCL encoder is already trained based on the entire graph. So, in this transductive learning setting, what would be the practical significance of the evasion attack? Could you perhaps provide a schematic or flowchart, similar to Algorithm 1 in your paper, that visually represents the process of the attack method in GCL? This could facilitate a better understanding of its operation and mechanics, thus enriching the comprehension of your methodology. 2. I recommend considering stronger graph injection attacks for evasion scenarios, such as those presented in [1]. 3. Could you provide a comparison between your proposed Randomized Edgedrop Smoothing (RES) method and the learnable edge-dropping augmentations described in [2]? I am interested in both theoretical distinctions and empirical differences based on experimental results. 4. Could you elucidate the distinction between your proposed Randomized Edgedrop Smoothing (RES) method and the conventional edge perturbation for augmentation as outlined in [3]? Understanding this difference could clarify the unique value proposition of your approach. 5. The definition of the "concatenation vector v" as given in line 106 seems somewhat ambiguous, as it appears to include the connected edge of node v. However, equation 5 also features h(v), where v presumably represents node features. Could you provide further clarification regarding the nature and role of the vector v? 6. In line 198, the space B is defined, but its details remain unclear. Could you provide a more comprehensive explanation of the space B, including its constituent elements? Is it meant to represent a probability space? [1]. Chen Y, Yang H, Zhang Y, et al. Understanding and improving graph injection attack by promoting unnoticeability[J]. arXiv preprint arXiv:2202.08057, 2022. [2]. Suresh S, Li P, Hao C, et al. Adversarial graph augmentation to improve graph contrastive learning[J]. Advances in Neural Information Processing Systems, 2021, 34: 15920-15933. [3]. You Y, Chen T, Sui Y, et al. Graph contrastive learning with augmentations[J]. Advances in neural information processing systems, 2020, 33: 5812-5823.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

NA

Reviewer ssR86/10 · confidence 3/52023-07-09

Summary

This paper studies the problem of certifiable robustness against adversarial perturbations in Graph Contrastive learning (GCL). This is an interesting paper where theoretical and empirical results are provided. The goal is to provide provable guarantee of robustness in the face of a limited budget adversarial attack on graph structure, showing that (1) positive samples remain positive samples in GCL, and (2) the downstream node/graph classification output does not change either. To this end randomized edge-drop smoothing (RES) is proposed and analyzed theoretically and empirically.

Strengths

The main advantage of this work is its certification capability, which eliminates dependence on empirical assessment relying on empirical attacks, whose parameter/algorithmic tuning would be questionable. The proposed method is simple and intuitive, and the experimental results show relative improvements on SOTA as well.

Weaknesses

The main disadvantage of the proposed work seems to be its reliance on the latent classes in the downstream task, which is considered to be node/graph classification. This still encompasses a large set of problems and is of interest, however, it does partially violate the claim that a general GCL certification is studied. Also, the manuscript has many typos and grammatical errors that would benefit from careful proof-reading.

Questions

1- In multiple occasions in the definitions and theorems, an assumption is made on the encoder h being a "well-trained" GNN encoder, however this term is not defined, and it is not clear what criteria should be met for this. Since this is repeated in multiple Theorems, its clarification is crucial. 2- Is the importance of certification of the GCL in 9 itself only theoretical, and a stepping stone to providing certification on the downstream node/graph classification? Or is it possible to also evaluate such certification? I cannot seem to find any such results in the experiments section. 3- In ablation study in 6.4, what was the optimal $\beta$ for the FLIP algorithm? The gap between FlIP and other methods (Baseline and Ours) seems too large, and it is interesting to know if the smaller \beta=0.1 turned out to be the best value, which would motivate having a finer grained search for a smaller value for this. 4- What is the rationale behind setting such high values for $\beta$ in experiments in Figure 2? Such high values indicated that almost all edges are dropped in the randomization, which means that no structural information is retained, which is counterintuitive given that such high *certified* accuracy can still be obtained.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Increase in computational needs for Monte Carlo estimates and its environmental impacts should be mentioned in the limitations section.

Reviewer 661C7/10 · confidence 4/52023-07-31

Summary

This paper is the first one to dive into the certifiably robust Graph Contrastive Learning (GCL) and proposes a certifiably robust GCL framework. It defines the certified robustness of GCL and then proposes Randomized EdgeDrop Smoothing (RES), which randomly drops each edge of the input sample with a certain probability. Besides, it also presents a simple training method for robust GCL. The theoretical analyses and extensive experiments demonstrate its effectiveness.

Strengths

This paper is the first one to dive into the certifiably robust Graph Contrastive Learning (GCL) and proposes a certifiably robust GCL framework. It defines the certified robustness of GCL and then proposes Randomized EdgeDrop Smoothing (RES), which randomly drops each edge of the input sample with a certain probability. Besides, it also presents a simple training method for robust GCL. The theoretical analyses and extensive experiments demonstrate its effectiveness. The paper is well-written.

Weaknesses

I have some minor comments, as below. 1. In Line 106, the definition of " concatenation vector \mathbf{v}" is confusing. What is the meanings of "which captures the upper triangular part of the adjacency matrix for the K-hop subgraph of v". Does it mean the K-hop neighbors of node v? 2. The use of probability operator symbols is inconsistent. For example, Eq. 2, 6, 7 use "\mathbb{P}", while Eq. 5 use "Pr". It is better to harmonize the symbols used.

Questions

Please refer to the weaknesses above.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

I believe this work does not have potential negative societal impacts.

Reviewer vCmM2023-08-16

Thanks for the response. I've read the rebuttal and I'd like to slightly raise my score.

Reviewer A65y2023-08-17

Official Comment by Reviewer A65y

I appreciate the authors' responses and additional experiments. The authors have addressed my concerns. I will keep my score the same.

Reviewer F8a72023-08-17

Thank you for the authors' response. Regarding Q1, are you suggesting that by using RES $v'$ still obtains a robust representation after $h$? For Q3 and Q4, I would like to gain a deeper understanding of the role of RES during both the training and inference phases. Which part contributes to the robustness? In line 373 of your paper, could you clarify what is meant by 'the number of removed structures'?"

Authorsrebuttal2023-08-17

Response to Reviewer F8a7

Thanks for your time and effort in reviewing our paper. Here is our point-to-point response to your further questions: **Q1: are you suggesting that by using RES, $\mathbf{v}’$ still obtains a robust representation after $h$.** Thanks for your insightful question. We kindly claim that given the perturbed sample $\mathbf{v}’=\mathbf{v}\oplus\delta$, where $||\delta||_0<k$, by using RES in the GCL training and inference phases, the representation of $\mathbf{v}’$ learned by our RES-based encoder $h$ can achieve certifiably robust performance in downstream tasks, that is, $h(\mathbf{v}’)$ has consistently correct prediction in downstream tasks under any perturbation within budget $k$. **Q3 and Q4: The role of RES during both the training and inference phases. Which part contributes to the robustness?** Thanks for your insightful question. We kindly claim that RES works both in the training and inference phases, which contributes to the robustness together. **The role of RES in the inference phase:** RES performs randomized edgedrop smoothing in the inference phase through Monte Carlo. Our motivation is that injecting the randomized edgedrop noise $\epsilon$ to $\mathbf{v}’$ in the inference phase for multiple times will make each perturbed edge in the majority of these samples will be dropped, making these perturbed edges finally drop in the final decision. which makes certifying the robustness of GCL feasible and promises $\mathbf{v}’$’s robust performance. Specifically, based on Sec. 5.2, RES will draw $\mu$ samples of $\mathbf{v}’\oplus\epsilon$. Then we use Monte Carlo to decide the final connection status of the edge by validating if the majority of the $\mu$ samples drop this edge. Thus, **all fake edges in most of the $\mu$ samples will be dropped if selecting high $\beta$**, making these fake edges finally dropped. **The role of RES in the training phase:** RES is proposed to train a robust encoder in the training phase to eliminate the negative impacts of randomized edgedrop noise in inference, and enhances model utility and robustness of GCL. Specifically, our motivation is that applying RES to test samples in the inference solely may hurt performance in downstream tasks and the certified robustness based on Eq.(8). Thus, we propose robust encoder training for RES in Sec. 5.1 by injecting randomized edgedrop noise into one augmented view during GCL. It ensures the samples with randomized edgedrop noises align in latent class with clean samples under the encoder, thereby mitigating the negative impacts of such noises and further benefiting the robustness and certification of RES. **Which part contributes to the robustness:** According to the claim above, RES works both in the training and inference phases, which contributes to the robustness together. In the inference phase, our randomized edgedrop smoothing can drop all fake edges if selecting a high $\beta$ to ensure certifiably robust performance under any perturbation within the specific attack budget. In the training phase, our robust encoder training method mitigates the negative impacts of randomized edgedrop noise in inference and enhances the model utility of GCL, further benefiting the robustness and certification of RES. **The meaning of 'the number of removed structures' in line 373.** Thanks for your question. The number of removed structures denotes the number of views/graphs where all edges are removed. In the training phase, we usually have two views for GCL training. In the inference phase, we usually have one test graph. Therefore, we use $i\in\\{0,1,2\\}$ and $j\in\\{0,1\\}$ to denote the number of removed structures in the training and testing phases to implement several variants of our model in ablation studies to understand how RES contributes to the robustness of GCL. We again thank the reviewer for the time and effort in reviewing our paper. If you have any further concerns or questions, please do not hesitate to let us know. We will respond to them timely.

Reviewer ssR82023-08-21

Response to authors

Thank you for responding to the previous questions. Most of my questions are answered, however I am still not convinced with the qualitative definition of "well-trained" as also in the above response, it still remains unclear what is meant mathematically (e.g. in terms of minimum desired classification accuracy, or such quantities.) If no such requirement is necessary for the GNN encoder accuracy for the theorems to hold, I'd recommend to drop the term "well-trained" in the theorems to avoid being qualitative and mathematically unclear. I will keep my score unchanged.

Authorsrebuttal2023-08-21

Response to Reviewer ssR8

Thanks for your kind response. Please see the following clarification for your question about the definition of the well-defined GNN encoder: To evaluate whether a GNN encoder $h$ is well trained or not mathematically, we introduce criteria based on the similarity between node/graph representations in the latent space. For each positive pair $(\mathbf{v}, \mathbf{v}^+)$ with its negative samples $\mathbf{V}^- = \\{\mathbf{v}\_1,\cdots,\mathbf{v}\_n\\}$, we clarify that $h$ is well-trained at $(\mathbf{v},\mathbf{v}^+)$ if the following inequality is satisfied: $$s(h(\mathbf{v}),h(\mathbf{v}^+))>\max_{\mathbf{v}^-\in\mathbf{V^{-}}}{s(h(\mathbf{v}),h(\mathbf{v}^-))},$$ where $s(\cdot,\cdot)$ is a cosine similarity function. This implies that $h$ can effectively discriminate $\mathbf{v}$ from all its negative samples in $\mathbf{V}^-$ and learn the meaningful representations for $\mathbf{v}$ in the latent space. Moreover, in this paper, we further extend the criteria for certifying robustness in GCL, as defined by Eq. (3). Based on the clarification above, given a GNN encoder $h$ that is well-trained at $(\mathbf{v}, \mathbf{v^{+}})$, suppose that $\mathbf{v}′$ is a perturbed sample obtained by adding structural noise $\delta$ to $\mathbf{v}$ as described in line 192, where $||\delta||\_0\leq k$. We then clarify that $h$ is certifiably robust at ($\mathbf{v}, \mathbf{v^{+}}$) if the following inequality is hold: $$s(h(\mathbf{v}{'}),h(\mathbf{v}^+))>\max_{\mathbf{v}^-\in\mathbf{V^{-}}}{s(h(\mathbf{v}{'}),h(\mathbf{v}^-))},~\forall{\delta}:\|\delta\|_{0} \leq k,$$ which indicates that for any perturbation within the attack budget $k$, the cosine similarity $s(h(\mathbf{v}{'}),h(\mathbf{v}^+))$ is consistently larger than $s(h(\mathbf{v}{'}),h(\mathbf{v}^-))$ for any $\mathbf{v}^-\in{\mathbf{V}^-}$. We sincerely thank the reviewer again for your time and efforts in improving our paper. We will include the above discussion in the final version of our paper. If you have any further concerns or questions, please do not hesitate to let us know. We will respond to them timely.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC