ARC: A Generalist Graph Anomaly Detector with In-Context Learning

Graph anomaly detection (GAD), which aims to identify abnormal nodes that differ from the majority within a graph, has garnered significant attention. However, current GAD methods necessitate training specific to each dataset, resulting in high training costs, substantial data requirements, and limited generalizability when being applied to new datasets and domains. To address these limitations, this paper proposes ARC, a generalist GAD approach that enables a ``one-for-all'' GAD model to detect anomalies across various graph datasets on-the-fly. Equipped with in-context learning, ARC can directly extract dataset-specific patterns from the target dataset using few-shot normal samples at the inference stage, without the need for retraining or fine-tuning on the target dataset. ARC comprises three components that are well-crafted for capturing universal graph anomaly patterns: 1) smoothness-based feature Alignment module that unifies the features of different datasets into a common and anomaly-sensitive space; 2) ego-neighbor Residual graph encoder that learns abnormality-related node embeddings; and 3) cross-attentive in-Context anomaly scoring module that predicts node abnormality by leveraging few-shot normal samples. Extensive experiments on multiple benchmark datasets from various domains demonstrate the superior anomaly detection performance, efficiency, and generalizability of ARC.

Paper

Similar papers

Peer review

Reviewer B6wn7/10 · confidence 5/52024-07-03

Summary

The paper produces a "one-for-all" generalist graph anomaly detection model. The proposed model first use feature projection and smoothness-based ranking to align the features of multiple datasets; then it employs a residual graph neural network to extract representations; finally, a cross-attention module is used to calculate the anomaly scores of query nodes. The paper is well motivated and with sufficient experiments for effectiveness validation.

Strengths

1. The setting of generalist graph anomaly detection is interesting and novel. The research question in this paper has high practice value. 2. The discussion of feature smoothness and abnormality is insightful, which may also inspires future studies. 3. The proposed method is carefully designed with three powerful modules. Each of them plays a critical role in capturing anomaly-aware knowledge and making precise prediction. 4. The experimental results show that the proposed method has significant performance advantage over the baselines. 5. The visualization experiment is interesting, which show the working mechanism of the attention module.

Weaknesses

1. The methodology details of the proposed method are not precise: specifically, which type of feature projection method is used in the experiment should be given. 2. The definitions of the concept of single-class normal and multi-class normal are missing, which hinders the readers to understand the design motivation in Sec. 4.3. 3. The figure is not readable enough. More annotations can be provided in Figure 2, especially for the R and C blocks.

Questions

1. In Eq. 6, the anomaly label is denoted by a bold letter y. Does that mean it's a vector? However, should it be a value actually? 2. What is the reason for the unstable anomaly detection performance under different number of context nodes?

Rating

7

Confidence

5

Soundness

4

Presentation

3

Contribution

4

Limitations

I believe the authors have addressed the limitation, and I haven't found any negative societal impact of this work.

Reviewer VWTk7/10 · confidence 4/52024-07-10

Summary

This paper proposes a new framework for graph anomaly detection called ARC. ARC uses in-context learning to detect anomalies across various graph datasets without requiring retraining or fine-tuning. It leverages few-shot normal samples during inference to achieve superior performance in anomaly detection tasks.

Strengths

Overall, this paper explores an interesting research topic with a well-designed method. The experimental results are also promising. 1. Methodology design. The design of the proposed method ARC is reasonable, where each component address a challenge in generalized graph anomaly detection. 2. Promising results. The experimental results show that indeed the ARC model can achieve better performance on a series of datasets.

Weaknesses

However, I have several concerns about this paper. 1. In the residual GNN component, a shared MLP is used for feature transformation, which is not vey commonly seen in other GNNs. Could you provide more intuition behind for explanation? 2. In attention mechanism, "value" is an important component in the computational process. However, the in-context learning module doesn't include the computation of "value". Can you give more detailed intuition for this design?

Questions

I still have some minor questions. 1. How to define the projected dimension $d_u$? 2. Considering the cost of fine-tuning (figure 6), will it require a lot of epochs to get the fine-tuned results for the baseline methods?

Rating

7

Confidence

4

Soundness

4

Presentation

3

Contribution

3

Limitations

The limitations are upfront in the paper.

Reviewer bLtA8/10 · confidence 4/52024-07-10

Summary

This paper investigates the research problem of "generalist graph anomaly detection (GAD)", targeting to address the cost and generalizability issues of the conventional GAD paradigm. This paper proposes ARC, a "one-for-all" GAD model that is pre-trained on a group of datasets and able to detect anomalies on new datasets on-the-fly. To verify the performance of ARC, experiments on GAD benchmark datasets are executed. The main contributions of this paper are: 1. Addressing a new research problem (generalist GAD); 2. Developing a novel method; 3. Conducting experiments for GAD performance validation.

Strengths

Originality: the proposed method has great originality. The three components in ARC are fresh and novel. The authors further discuss the difference and connection between ARC and other method, which is sufficient to show the originality of this paper. Quality: This paper is of high quality. The research problem and proposed method are both solid. Comprehensive experiments and discussions are given in the paper (some are in the appendix). Clarity: The presentation of this paper is good, making it easy to read. Most of the equations, description, figures, and tables are good to understand. Significance: I'm not an expert of GAD domain, but I think this paper make good significance in the community. Considering the significant difference between the proposed generalist GAD paradigm and the existing one, I think think paper is an important step towards AGI.

Weaknesses

Methodology details: Some specific details of the method design are not clearly given. In specific: 1. For the feature projection, what if the original feature dimension is smaller than the predefined projected dimension? 2. Why doesn't ARC use the value-based cross attention? Is there any disadvantage? Experiment details: Although the experiments are extensive, I think more discussions should be given for some notable results. In specific: 1. Why unsupervised methods perform better than the supervised methods? 2. Why Figure 5 (b) has a dropping trend at nk=20?

Questions

The questions listed in Weaknesses are expected to be answered. Meanwhile, I have some extra questions: 1. Why "pre-train & fine-tune" settings only include unsupervised methods? 2. Will ARC be time-costly when the number of context nodes become large?

Rating

8

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

From my perspective, the authors have addressed the limitations and the paper seems doesn't have potential negative impact.

Reviewer HCAf4/10 · confidence 4/52024-07-15

Summary

This paper proposed AARC, a generalist GAD approach to detect anomalies across various graph datasets on the fly. It consists of feature alignment module, residual graph encoder and in-context anomaly scoring module.

Strengths

1. The experimental results show that the proposed method outperform most of the baseline methods across various datasets. 2. The presentation of this paper is good and the paper is easy to follow.

Weaknesses

1. The smoothness-based feature sorting is done in the raw input feature. In this case, after unify the feature dimensionality with the feature projection, how do you smooth the new feature $\hat{X}^{(i)}$ with smoothness-based feature sorting? In another word, how do you aligns dimensionality and smooth feature at the same time? 2. In line 232 and 233, the authors mention that "In the first two steps, we perform propagation on the aligned feature matrix $X'=X^{[0]}$ for $L$ iterations, and then conduct transformation on the raw and propagated features with a shared MLP network". This is confusing. It's unclear weather the input feature is aligned feature matrix or the raw feature. Besides, the aligned feature matrix should be $\tilde{X}$ rather than $X$. 3. Equation 3 can be written as $Z^{[l]}=MLP(\tilde{A}X^{[l-1]})=\sigma(\tilde{A}X^{[l-1]}W)$, where $W$ is the weight matrix of MLP and $\sigma$ is the activation function. In this case, what's the difference between equation 3 and traditional GNN. The claim of equation 3 to capture the high-frequency signals and local heterophily is not convincing. 4. The design of cross-attentive in-context anomaly scoring is based on the assumption that the normal query have similar patterns to several context nodes and their embeddings can be easily represented by linear combination of context node embeddings. However, in many cases, given the limited number of normal nodes, especially for the test graph, it's highly likely that some unsampled normal nodes cannot be represented by linear combination of other normal nodes. For instance, the sampled normal nodes from class 1 to class 5, while there exist some unsampled normal nodes from class 6. In this case, these unsampled normal nodes may not be represented by the linear combination of other normal nodes and these nodes tend to have large reconstruction error, thus being labeled as the abnormal nodes. How do you mitigate this issue?

Questions

1. The smoothness-based feature sorting is done in the raw input feature. In this case, after unifying the feature dimensionality with the feature projection, how do you smooth the new feature $\tilde{X}^{(i)}$ with smoothness-based feature sorting? In another word, how do you aligns feature dimensionality and smooth feature at the same time? 2. The design of cross-attentive in-context anomaly scoring is based on the assumption that the normal query have similar patterns to several context nodes and their embeddings can be easily represented by linear combination of context node embeddings. However, in many cases, given the limited number of normal nodes, especially for the test graph, it's highly likely that some unsampled normal nodes cannot be represented by linear combination of other normal nodes. For instance, the sampled normal nodes from class 1 to class 5, while there exist some unsampled normal nodes from class 6. In this case, these unsampled normal nodes may not be represented by the linear combination of other normal nodes and these nodes tend to have large reconstruction error, thus being labeled as the abnormal nodes. How do you mitigate this issue? 3. For the supervised baseline methods, what is the available label information for the test graphs. Based on the setting in the paper, it seems that only a small percentage of normal nodes are available for training for test graphs. In this case, how do you train these supervised baseline methods? What is the training procedure for these baseline methods?

Rating

4

Confidence

4

Soundness

2

Presentation

3

Contribution

2

Limitations

The authors list limitations in conclusion.

Authorsrebuttal2024-08-11

Reply to the further questions raised by Reviewer HCAf

We are grateful to Reviewer HCAf for the valuable feedback. Please find our responses to your new questions below. We hope that our response addresses your concerns. **Q6: Impact of unseen normal class problem** **A6:** We appreciate you once again for highlighting the unseen normal class problem. We agree that a deeper discussion on the impact of this issue is necessary. As shown in Fig. 5 and Fig. 9, the performance gaps between $n_k=2$ (where normal classes should not be fully covered) and $n_k=100$ (where normal classes should be covered) are generally within the range of $0.5$% - $5$% in terms of AUROC. This indicates that the unseen normal class problem may be a factor affecting the performance of ARC. However, we would like to note that the unseen normal class problem is not a fatal issue of ARC, since the performance when $n_k=2$ remains generally acceptable and won't collapse to indistinguishable prediction (e.g., AUROC=50% and AUPRC=0%). We want to express our appreciation to the reviewer once again for identifying the potential unseen normal class problem in ARC. We will definitely discuss the potential limitations caused by this issue in the revised manuscript and provide possible strategies (in A4) for users/readers to mitigate this problem. **Q7: Codes of propagation in ARC** **A7:** Thank you for your detailed observation regarding the implementation of the Ego-Neighbor Residual Graph Encoder in the ARC model. The reason the adjacency matrix is not directly used in the forward function (Lines 21-39 in model.py) is due to our decoupled architecture in Eq.(3), i.e., the propagation is directly conducted on the features without learnable parameters. In this case, we can move the propagation step into the codes of data preprocessing to enhance the running efficiency (since we can only run it once). Specifically, the codes of propagation are in the "propagated" function (Lines 143-148 in utils.py), which is called in Lines 53-57 in main.py before the training of the encoder. Then, the propagated features (saved in Dataset.graph.x_list) are subsequently used in the forward function (Lines 21-39 in model.py) for downstream residual encoding. We appreciate your suggestion, and to prevent potential misunderstandings, we will incorporate more comments in the publicly released code for interpretation.

Reviewer HCAf2024-08-13

Reply to Authors' response

Thanks for the detailed reply. I am still concerned about the generalization of the proposed method as the generalization capability is the major contribution in this paper. I agree with the comment by reviewer **RwhU** that "The method section did not provide a solid theoretical analysis to support this claim." when dealing with the new dataset. In addition, the experimental results are not sufficient to demonstrate the capability of the proposed method on the unseen tasks. In addition, I have one more question regarding your response to Reviewer **RwhU** about the results of few-shot samples. Why does the performance of ARC achieve the best performance with limited number of pseudo samples (e.g., #Pseudo Normal = 2) on some datasets, such as Amazon, Facebook and Weibo? As you mention during the rebuttal that "the advantage of having a larger number of normal samples reduces the impact of anomalies within the pseudo normal samples.", while the performance of ARC tends to decrease on these datasets when the number of pseudo normal samples increases. Do you have any explanation or thoughts to this observation?

Authorsrebuttal2024-08-14

We genuinely appreciate your thoughtful questions. Before we address your subsequent questions, could you confirm if our response has addressed your concerns regarding **Q1-Q7**? We want to ensure that all your concerns have been adequately addressed before we discuss any new questions. If there are any aspects you feel need further clarification or improvement, please do not hesitate to let us know. The responses to your new questions are given as follows. **Q8: Generalization capability of ARC** **A8:** From a theoretical perspective, ARC is equipped with an in-context learning (ICL) technique, which enhances its generalization capability on new datasets. Specifically, the ICL mechanism enables ARC to distinguish between normal and abnormal samples by using a set of normal samples from each given testing dataset, rather than fitting to the anomaly patterns of a specific training dataset. Following the assumption that normal samples are more similar to each other than anomalies, the cross-attention-based ICL module can learn to make accurate predictions by leveraging inter-sample similarity on new datasets. From an experimental perspective, we have made every effort to demonstrate the generalizability of ARC through experiments on datasets from a wide range of domains. Specifically, in the original manuscript, we included 8 datasets from 3 domains: **citation network, social network, and co-review network**. During the review process, we further expanded our experiments to cover 4 additional domains: **co-author network, co-purchase network, transaction record network, and work collaboration network**. It is important to note that our model had **never seen** data from these 4 new domains during training, providing strong empirical evidence of ARC’s generalization capabilities on diverse and unseen tasks. To the best of our knowledge, we have covered datasets from most of the mainstream domains in graph anomaly detection (GAD), including **4 datasets for training and 12 datasets for testing**, which exceeds the evaluation scope of the vast majority of GAD studies. **Q9: Results discussion of experiments of pseudo normal trick** **A9:** Regarding your question about the performance of ARC with a limited number of pseudo samples, we would like to clarify the following points: **Optimal number of pseudo samples.** The statement “Why does the performance of ARC achieve the best performance with limited number of pseudo samples (e.g., #Pseudo Normal = 2) on some datasets, such as Amazon, Facebook and Weibo?” is not accurate. As mentioned in our response to Reviewer RwhU, ARC achieves its best performance on these datasets with #Pseudo Normal = 10. This is because, in real-world scenarios, the number of normal samples typically far exceeds the number of anomalies. Therefore, with a sample size of 10, the probability of selecting true normal samples is higher, which is why we set the default sample size to 10 in all our experiments. This demonstrates that ARC can achieve competitive performance with a small number of shot samples. **Impact of sample size on performance.** The statement "the advantage of having a larger number of normal samples reduces the impact of anomalies within the pseudo normal samples" was intended to illustrate the intuitive feasibility of the "pseudo normal trick." However, this does not imply that increasing the number of pseudo normal samples will always enhance ARC’s performance. On the contrary, when the sample size is very small (e.g., $n_k$=2 or 10), the probability of selecting true normal samples is higher. As the sample size increases, the likelihood of including false normal samples also increases (especially when the anomaly ratio of the dataset is relevantly large), introducing more noisy psuedo samples and potentially degrading performance. To sum up, the performance of ARC with the "pseudo normal trick" is influenced by the "purity" of the sampled pseudo examples, which in turn depends heavily on the anomaly ratio of the dataset and experimental randomness. Therefore, it is reasonable that different datasets exhibit varying preferences for the pseudo sample size. Last but not least, we'd like to clarify that **the exploration of the pseudo-label trick is merely an additional exploratory experiment**. We aimed to investigate the possibility of applying ARC to a purely unsupervised (zero-shot) scenario. Surprisingly, the results show that the performance is still acceptable, indicating ARC's potential. However, our main focus remains on the few-shot setting, where reliable content samples are available for ICL. We hope this experiment demonstrates the potential of ARC and inspires future work in this direction, but it is not the main contribution of this paper.

Authorsrebuttal2024-08-14

Finally, we express our sincere appreciation for your thorough review and insightful questions. We want to emphasize that this paper introduces a new paradigm in graph anomaly detection (GAD), known as generalist GAD, which presents a greater challenge than conventional GAD problems. Additionally, the proposed method is fundamentally different from traditional GAD models, incorporating well-crafted techniques for data alignment and in-context learning. We hope that this frontier discovery of “one-for-all” GAD models will inspire the development of innovative and more generalized solutions within the community. We kindly ask you to review our explanations with an open mind, considering how our method diverges from conventional GAD approaches. Your expertise is invaluable, and we would be grateful if you could reassess our work in light of these clarifications. Given this new perspective, we respectfully request that you consider providing a fair evaluation that reflects the true nature and potential impact of our research. Your impartial assessment would be greatly appreciated. Thank you for your time and consideration.

Reviewer RwhU4/10 · confidence 4/52024-07-24

Summary

The paper introduces an ARC method designed to detect anomalies across diverse graph datasets without the need for dataset-specific training. ARC leverages in-context learning with few-shot normal samples during inference, comprising three main components: a smoothness-based feature alignment module, an ego-neighbor residual graph encoder, and a cross-attentive in-context anomaly scoring module. The model demonstrates superior performance, efficiency, and generalizability compared to existing GAD methods, as validated by extensive experiments on multiple benchmark datasets.

Strengths

1. Graph anomaly detection is very challenging. This paper untangles this problem into three folds and solves them one by one, making the paper technically sound. 2. The proposed ARC model is a “one-for-all” GAD model that is capable of identifying anomalies across target datasets from diverse domains, without the need for re-training or fine-tuning.

Weaknesses

1. The paper's claim that it is "capable of detecting abnormal nodes across diverse graph datasets from various application domains without any training on the specific target data" is a bit ambitious. The method section did not provide a solid theoretical analysis to support this claim. 2. The statement in Line 102 should be moved to the introduction section to declare the scope of this work. 3. The datasets used in the experiments are somewhat similar. It is recommended to include more diverse datasets from different backgrounds. 4. The model relies on the availability of a few-shot normal samples for in-context learning. In some practical situations, obtaining these samples may be challenging, particularly in highly anomalous or unknown environments.

Questions

see the weaknesses above

Rating

4

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

yes

Authorsrebuttal2024-08-13

Dear Reviewer RwhU, We sincerely appreciate your time and expertise in reviewing our submission. Acknowledging the demands on your schedule, we are mindful not to intrude on your time. However, we would be grateful if you confirm that our rebuttal adequately addresses your concerns. Thank you in advance for your consideration. Authors

Reviewer HCAf2024-08-11

Reply to authors' rebuttal

I thank the authors for their thoughtful rebuttal. Most of my concerns have been properly addressed, but some remain: - For the answer to question 4, though the experimental results in Table 1 on the Cora dataset show that the proposed methods significantly outperform the other baseline methods, it can be also observed that some baseline methods can outperform ARC on other datasets, such as Facebook, Weibo. Specifically, TAM (Unsupervised - Pre-Train & Fine-Tune) outperforms ARC by around 7% on Facebook dataset. It's not a consistent observation throughout all evaluated datasets. Thus, I am not fully convinced by the statement that this issue does not affect the performance a lot. - I look at the code of ARC. The forward function in ARC class (Line 21 -Line 39 in model.py file) corresponds to the implementation of Ego-Neighbor Residual Graph Encoder, I find that **the adjacency matrix is not used for representation learning, which is not consistent with Equation 3**.

Reviewer bLtA2024-08-13

Thanks for the authors' rebuttal, which addresses all my concerns. I would like to raise my rating in response to the authors' efforts during the rebuttal.

Reviewer B6wn2024-08-14

Dear Authors, Thanks for the detailed responses, which fully addresses my previous concerns. I decide to maintain my positive scoring for this paper.

Reviewer VWTk2024-08-14

Thanks for the rebuttal. After reading all the comments, I decided to keep my rating of accept.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC