DeTeCtive: Detecting AI-generated Text via Multi-Level Contrastive Learning

Summary

This paper presents DeTeCtive, a new algorithm for detecting AI-generated text. The key insight in this paper is that instead of treating AI-generated text detection as a binary classification problem, it should be treated as a multi-class "author style" classification problem. Using this insight, the authors develop a contrastive learning algorithm which learns style vectors for each LLM style, which are then stored in a style vector database for inference via KNN classification. The learnt style vector database can easily be expanded at inference to accomodate new text generated from potentially OOD LLMs. The authors perform experiments on 3 existing datasets with outputs from 8 to 27 different LLMs, and show that DeTeCtive outperforms some existing methods. The authors complement their work with ablation studies on their loss function, and analysis experiments in OOD settings.

Strengths

1. The paper presents an interesting reformulation of AI-generated text detection, as a multi-label "LLM style" classification problem. The paper presents an intuitive contrastive learning algorithm to learn style vectors for each LLM style, which are then stored in a style vector database for inference. 2. It seems to be easy to adapt the proposed algorithm towards OOD LM-generated text. By simply computing vector representation on an OOD LLM's generated text, the algorithm can support AI-generated text detection on this OOD LLM in future cycles. The authors perform analysis experiments to confirm its effectiveness. 3. While I do have numerous concerns with the experiments, the authors do make an attempt at a large-scale empirical evaluation of their method, considering 3 existing datasets with outputs from 8 to 27 different LLMs.

Weaknesses

I had some concerns about the experiments in this paper. 1. **Baselines seem to be quite weak / over a year old**. Most of the baselines used are either pre-2022 methods or DetectGPT, which seems unsuitable for a cross-model dataset like the one used in evaluation. How do newer methods like Binoculars [5] perform on the same benchmark? Also, why not compare against watermarking algorithms like KGW [1], EXP-Edit [2], SemStamp [3], or [4]? Also how well do commercial tools like GPTZero perform on the same datasets (https://gptzero.me)? 2. **Limited experiments on newer GPT-4 class LLMs**. AI-generated text detection is the hardest on the most human-like LLMs, which are the GPT-4 class models. However, almost no experiments were done on GPT-4 class models (except for a small subset of the M4 dataset). I encourage the authors to stress test their OOD generalization to outputs from GPT-4 class models like Claude Opus, Gemini 1.5 and the newer variants of GPT-4. 3. **Limited emphasis on attack robustness**. There are almost no experiments in the paper (besides the mention of OUTFOX paraphrases) showing the robustness of the DeTeCtive algorithm to paraphrasing [6, 7], text mixing attacks [8], and translation attacks. This is critical to establish the robustness of the method compared to alternative detectors. 4. **It's not clear whether gains over baselines are coming from the SimCSE initialization or training**. Looking at the ablation studies in Table 3 and baselines in Table 1, all ablated variants outperform all baselines. This makes me question the need for the complexity in the DeTeCtive method's loss function, and how much of the gains can be attributed to the SimCSE initialization rather than the proposed DeTeCtive algorithm. An ablation study that would help here is using the DeTeCtive loss function on alternative encoders like RoBERTa / BERT / SentenceBERT. [1] - https://arxiv.org/abs/2301.10226 [2] - https://arxiv.org/abs/2307.15593 [3] - https://arxiv.org/abs/2310.03991 [4] - https://arxiv.org/abs/2305.08883 [5] - https://arxiv.org/pdf/2401.12070 [6] - https://arxiv.org/abs/2303.11156 [7] - https://arxiv.org/abs/2303.13408 [8] - https://openreview.net/pdf?id=DEJIDCmWOz ---- **After rebuttal**: I've decided to increase my score from 3 to 5 due to extra experiments on paraphrase robustness, and baseline comparisons against Binoculars.

Questions

1. AI-generated text detectors typically need to operate in a low FPR setting, to minimize the risks of labeling innocent text as AI-generated. Given this, what are the Table 1 true positive rates at a low FPR of say 0-1%? TPR at low FPR ranges (0-1%) is a standard metric for evaluating AI-generated text detectors which has been used in many previous papers and blogs: https://arxiv.org/abs/2303.13408, https://openreview.net/pdf?id=DEJIDCmWOz, https://arxiv.org/pdf/2401.12070, https://foundation.mozilla.org/en/blog/who-wrote-that-evaluating-tools-to-detect-ai-generated-text/

Rating

5

Confidence

4

Soundness

2

Presentation

3

Contribution

2

Limitations

The authors acknowledged weakness #3 in their limitations section. But I think this is a critical limitation and analysis on paraphrase / attack robustness is necessary in AI-generated text detection papers.

Summary

This paper proposes to learn a text encoder with contrastive learning to cluster texts from different sources for fine-grained classification. A training-free incremental adaptation method is designed for detecting OOD data.

Strengths

1: The innovation of TFIA is inspiring for OOD detection. Detection in information retrieval style is interesting and exhibits great performance. 2: The solid experiments across multiple datasets prove the effectiveness of the proposed method.

Weaknesses

1: The proposed multi-level contrastive loss shares some similarities with existing work [1] so it is not exciting enough. 2: Although the authors conduct extensive experiments on multiple datasets, only few of them are designed specifically for AI text detection (GLTR, DetectGPT). They should provide more comparisons with SOTA methods [2,3]. 3: As a work towards detecting AI-generated texts, more analysis of the features of texts should be provided. Otherwise, the framework could be applied to any binary classification problem (with hierarchical labels). [1] Liu S, Liu X, Wang Y, et al. Does\textsc {DetectGPT} Fully Utilize Perturbation? Selective Perturbation on Model-Based Contrastive Learning Detector would be Better[J]. arXiv e-prints, 2024: arXiv: 2402.00263. [2] Verma V, Fleisig E, Tomlin N, et al. Ghostbuster: Detecting text ghostwritten by large language models[J]. arXiv preprint arXiv:2305.15047, 2023. [3] Chen Y, Kang H, Zhai V, et al. Gpt-sentinel: Distinguishing human and chatgpt generated content[J]. arXiv preprint arXiv:2305.07969, 2023.

Questions

1: I am wondering if this framework would do a good job in authorship attribution [1]? 2: For the classification approach in ablation studies, is the classification head trained with your own data or initialized in other ways? 3: Could you please provide some explanation on the advantage of multi-level contrastive loss over pcl? It is confusing to me that fine-grained clustering could help with binary classification. [1] Uchendu A, Le T, Shu K, et al. Authorship attribution for neural text generation[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020: 8384-8395.

Rating

5

Confidence

4

Soundness

2

Presentation

2

Contribution

2

Limitations

The authors mention they do not explore the robustness of the proposed method. The validation could be conducted under different attacks like paraphrasing, editing, prompting.

Summary

The paper discusses the challenges of current AI-generated text detection methods, which often suffer from performance issues and poor adaptability to new data and models. The authors introduce a new framework called DeTeCtive, which uses multi-level contrastive learning to distinguish different writing styles rather than just classifying text as human-written or AI-generated. This approach improves the effectiveness of various text encoders, achieving top results across multiple benchmarks, particularly in out-of-distribution scenarios. DeTeCtive's framework includes a dense information retrieval pipeline and a Training-Free Incremental Adaptation (TFIA) mechanism, enhancing performance without extra training when encountering new data. The method fine-tunes text encoders using a new multi-task auxiliary, multi-level contrastive learning loss to capture detailed features of different writing styles. Extensive experiments show that DeTeCtive surpasses existing methods in detecting AI-generated text and excels at handling data from unseen models and domains. The paper also emphasizes the method's compatibility with various text encoders and its strong performance in diverse scenarios, making a significant contribution to the field of AI-generated text detection and promoting the safe use of large language models.

Strengths

The use of multi-level contrastive learning to distinguish writing styles is a novel method that goes beyond traditional binary classification. This allows for a more nuanced detection of AI-generated text, improving the overall detection performance. The proposed method consistently outperforms existing techniques across multiple benchmarks, establishing new state-of-the-art results. This indicates the effectiveness of the multi-level contrastive learning framework.

Weaknesses

While the paper shows strong performance on the chosen benchmarks, it remains unclear how well the model would perform on other datasets or in different application contexts. This limits the generalizability of the findings to some extent.

Questions

How does the quality and diversity of the training data affect the performance of DeTeCtive? Are there specific datasets that are more beneficial for training the model?

Rating

7

Confidence

3

Soundness

3

Presentation

3

Contribution

2

Limitations

The focus on distinguishing writing styles might overlook content-based cues that could also indicate AI-generated text. This might lead to missed detections if an AI successfully mimics human writing style while generating misleading content.

Thank you for the detailed response and experiments, raising score to 5

Thank you for the detailed response and additional experiments! I've decided to increase my score from 3 to 5 due to extra experiments on paraphrase robustness, and baseline comparisons against Binoculars/GPTZero. I still encourage the authors to do add watermarking baselines in the paper, perhaps in the supplementary material.

Response

Thank you for the detailed explanation and additional experiments. Here are some after-rebuttal questions: * Q1 My major concern is that this work focuses too much on improving binary classification performance instead of providing insights about synthetic text, which deviates the research topic -- do you think SCL is targeted at detecting synthetic data even if it is a commonly used baseline (even a strong baseline) in this area? Many existing works about AI-generated text detection discuss the differences between AI-generated and human texts, which deepens communities' understanding about AI-generated texts. For example, DetectGPT finds the difference in log probability between two categories. GPT-Sentinal traces integrated gradient to tokens to decide which tokens contribute to the classification. CoCo discovers the coherence structure difference in two categories -- these are examples from the related work introduced in this paper. I agree that the visualization results prove the effectiveness of the learned encoder. However, I argue that this is a success in classification (I think SCL is also able to cluster the data embedding well) but not in explaining the intrinsic difference between AI-generated and human texts -- the research problem in this paper. This is a methodologically inspiring work. I lean to reject this paper for the weak relation to the problem it claims to solve based on the above-mentioned reason. I am willing to increase my score if the authors address my concern. * Q2 I have a question about the authorship attribution experiment. If I do not misunderstand, the author indicates the result is in Table 2 in the attached PDF, which is the same (except for the addition of Binoculars) in Table 1 in the original paper. I assume the results in table 1 in the original paper is for binary classification? But authorship attribution is a multi-class classification problem and I did not expect the same results between the two tables. Please tell me if I am wrong about anything.