Intrinsic Dimension Estimation for Robust Detection of AI-Generated Texts

Summary

This paper proposes a method to detect AI-generated texts based on intrinsic dimensions of sequences generated by humans and LLMs. The authors use a method called persistence homology dimension (PHD) to estimate intrinsic dimensions of both human- and LLM-generated texts. Using a variety of datasets across multiple languages, the authors first notice that intrinsic dimensions between human- and machine-generated texts tend to differ, with the intrinsic dimension of human texts being higher than that of machine-generated ones (the intrinsic dimensions again differ between languages, yet they are always differentiable from human texts). Using the estimated dimensions, the authors then train a simple single-feature logistic regression classifier to differentiate between human- and machine-generated texts. Experimental results reveal that their method substantially surpasses existing baselines (e.g., DetectGPT, GPTZero), and interestingly the method remains robust against paraphrase attacks conducted using DIPPER. Finally, the authors show that their method is more robust against samples authored by non-native speakers than existing baselines.

Strengths

* The authors propose a novel method to detect AI-generated content, outperforming existing works. * The method is effective yet easy to understand, and I believe researchers working on similar topics would be quite interested in the presented results. * The additional experiments showing that the model is robust against paraphrase attacks further strengthen the method’s relevance.

Weaknesses

* The analysis could be more extensive. For example, given that the classifier is based on a single feature, it would be interesting to see how performance varies as a function of dataset size, and how vulnerable its generalization capabilities are to ‘noisy’ datasets (i.e., those where dimensions for individual samples have high variance). While this has partially been explored (Figure 2), showing classification results in the Experiments section would have been informative. * It would be nice if Table 1 could be extended for additional approaches to ID estimation (line 188 mentions that 12 approaches have been explored). Showing results across those approaches (e.g., in a boxplot) would give the reader a better notion of how well the differences in dimensions between human- and AI-generated texts generalize across methods.

Questions

None

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The authors clearly outline the limitations of their work in a dedicated section. However, a few additional words on the broader impact (e.g., the applicability of the method in practice) would be helpful.

Summary

The paper describes work on detecting AI-generated text using intrinsic dimensionality (ID) estimation methods through Persistent Homology Dimension (PHD) (and MLE). The authors motivate this approach by highlighting that written texts between machine and humans observe differences in topological representation. The authors explored this method using an extensive set of experiments covering various SOTA models for detection, such as DetectGPT, GPTZero, etc., across multiple languages in a crosslingual setup and across multiple source domains of data such as Reddit, StackExchange, and Wikipedia. Results show that using ID as a feature generally performs consistently and somewhat robustly across other methods, especially in general detection. Overall, I believe the paper is a good addition to NeurIPS, especially as it gives a unique perspective on detecting machine-generated texts that’s theoretically motivated.

Strengths

There are a number of things that I liked about the paper that contributed to my overall recommendation. First, the paper gives a unique point-of-view to the standard AI-generated text detection task which deviates from the usual approach of looking at surface-based style and linguistic characteristics. The paper shows strong motivation and support as to why intrinsic methods such as PHD shows substantial differences between human-written and machine-generated texts. Second, I find merit in the breadth and depth of the experiments conducted which extends to crosslingual and crossdomain settings. It is also good that the authors explored a number of generative models and targeted some of the concerns with detecting machine-generated text such as errors with texts from non-native speakers. Lastly, the paper is readable enough for a larger variety of readers to appreciate its contributions. With these points, I recommend the paper for acceptance.

Weaknesses

There are no major issues in the paper that I’m particularly concerned about. There are some points, however, that I would recommend to be given more emphasis and clarification to improve the quality of the discussion as well as to increase confidence with the results. 1. I would have appreciated it better if the authors provided their clear inferences as to why the PHD-based detector, even with a paraphraser, is substantially better than the other classifiers. If the result indeed shows that the method is effective, what does it imply? Does it suggest that using a non-linguistic representation such as the intrinsic dimension of data compared to other linguistic factors considered by DetectGPT, RankGen, etc. is more practical? Quantitatively, how much offset in performance do paraphrasers like DIPPER affect the proposed classifier as well as the other detectors? It would also be interesting and a good addition in the discussion if the authors can show that even using paraphrasers, the proposed detector is still significantly effective through a statistical test. 2. There are a few vague statements in particularly focused on parts of the discussion of results. In the cross-domain and cross-model performance, statements such as “the PHD classifier is not influenced by domain transfer” and “PHD classifier to be more robust to entirely new AI models” need further clarification and even rephrasing. Achieving relatively favorable performance on three datasets hardly makes any model robust and immune to domain shift unless proven in-depthly with more data and/or backed by statistical tests. 3. It seems odd that ChatGPT was never mentioned anywhere in the statement of models used for experiments and then appears on one of the results section. Was this portion rushed? Justification is needed for this as readers would expect all experiments on evaluating generations to come from what was previously declared and should be uniform for all experiments. Also, there’s no “GPT3.5-davinci-003” in OpenAI’s model endpoints (https://platform.openai.com/docs/models/model-endpoint-compatibility). This may have been a typo so kindly clarify. 4. Same recommendation also applies where the authors should provided more discussion instead of describing the values on the table as seen in the experiment on non-native written text vs. AI. It was observed that existing models such as GPTZero and OpenAI tend to have higher rates of misclassifying non-native text as AI-generated but not so much for PHD and especially using MLE-based classifiers. Thus, what are the author’s insights as to why MLE performs better than PHD for this type of experiment? The authors previously mentioned that MLE performs well on “small samples, noisy settings, and small variance” so how does this ties up with the result? What gives MLE the edge over PHD in this scenario? 5. There are mentions of using specific splits of existing datasets in the paper. These should all be added as a separate table in the Appendix. It’s quite challenging as the reader to approximate how large the data used specifically for the study. This information is also used when analyzing the model performances.

Questions

Aside from the supporting questions highlighted in the Weaknesses section, here are some points that further require clarification: 1. Is it scientifically correct to refer to the intrinsic dimensionality methods as non-linguistic approach for detecting AI-generated texts? To me, the method seems to merit its own category compared to approaches using stylistic properties of texts. 2. What decoding hyperparameter values were used? Are these uniform across all model generation setups? 3. Is it possible that the training setup of RoBERTa contributed to it being more robust across various domains? RoBERTa’s performance was highlighted but the authors did not provide any of their own inferences.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

3 good

Contribution

4 excellent

Limitations

The added limitations section seems acceptable but I would recommend to further clarify the breadth of dataset used to merit claiming robustness and applicability to domain shift. It might be worth comparing to previous literature on how much data they used before they can refer to one model as robust.

Summary

This paper proposes a new method for artificial text detection(ATD) with intrinsic dimension (ID) estimation. First, contextualized representation of tokens in the text is extracted by a RoBERTa model. Next, the author estimates the ID of this set of contextualized representations : (1) N tokens and their corresponding vectors are sampled from the set, forming a vertex set; (2) the persistent homology dimension (PHD) is estimated by measuring the lengths of the edges in a minimal spanning tree with the vertex set; (3) varying the value of N to obtain a set of N-PHD pairs, and measuring the ID based on the slope of their linear correlation. Finally, the ID is utilized as a feature in a logistic regression model for binary classification. The author has conducted extensive experiments on widely-used benchmarking datasets. The results demonstrate that the proposed method exhibits greater robustness when compared to existing ATD methods, particularly in terms of its resilience towards adopted AI models, paraphrase attacks, and non-native speakers. Furthermore, in comparison to a conventional RoBERTa-based classifier, this method showcases significantly improved out-of-domain performance in both domain and AI model transfer scenarios. Additionally, the author has curated a new dataset for multilingual ATD, although specific details regarding this dataset are not provided.

Strengths

1. The application of persistent homology and intrinsic dimension estimation for ATD is both intriguing and well-founded in theory. While the methodology is similar to that of [1], which is already cited in the paper, this method relies on fewer features and conducts more comprehensive experiments to assess the robustness of competing methods. Hence, the proposed method and its findings exhibit a significant level of novelty. 2. The proposed method has achieved promising performance, particularly in terms of robustness. This is mainly because it does not need to finetune parameters of a large Transformer model, nor does it assume reliance on an AI model that is likely to generate the text. 3. Extensive experiments and analysis are done to demonstrate the effectiveness of the method. [1] Kushnareva, Laida, et al. "Artificial Text Detection via Examining the Topology of Attention Maps." EMNLP. 2021.

Weaknesses

1. Some claims are not well supported. The author claims that they estimate the intrinsic dimensions (ID) of text data. In fact, what they really estimate is the ID of the contextualized representations. By contrast, [1] directly estimates the ID of natural images. 2. Although extensive experiments have been conducted, one important analysis is missing: investigating the impact of the decoding algorithms. One important assumption of this work is that the metric space of human-written text has more isolated sub-graphs, which can be incurred by rapid shift of topic, usage of rare words, and so on. AI models can also achieve these characteristics by adjusting the hyperparameters of decoding algorithms, such as reducing the softmax temperature or increasing the values of P and K in nucleus and top-K sampling, respectively. As seen in Table 2, PHD has better performance for detecting GPT-3.5 than GPT-2, mostly because GPT-2 tends to generate text that has more grammar mistakes and less akin to human language. Consequently, the reliability of the method in the face of different sampling techniques becomes questionable. 3. The paper is not very easy to follow: some important details and definitions are missing. To name a few: * Some background information about topological features, especially connected components, are necessary to understand the estimation of persistent homology dimension. * Lack of definition of $C$ in Line 177, as well as $\tilde{C}$ in Line 183. * In Section 3, the author solely explains the estimation process for $E^{i=0}$ without explicitly indicating that they set $i=0$ until Section 4. It would be clearer if this information was explicitly stated earlier. * The author lists the new dataset as one of the main contributions of their work. However, no specific details are presented. It would be helpful to know the dataset's statistics, the curation process employed, as well as any annotation, cleaning, or pre-processing steps that were carried out. [1] Pope, Phil, et al. "The Intrinsic Dimension of Images and Its Impact on Learning." ICLR. 2020.

Questions

## Typos 1. Line 45: some downstream task(s) 2. Line 177: where equivalence mean(s) ## Suggestions 1. It would be helpful to include some text examples with high and low ID. ## Questions 1. Why RoBERTa-large has higher variance than RoBERTa-base(Figure 5) 2. What does the * mean in Table 2? 3. Why do you shuffle the order of datasets in Table 3? It is weird that PHD has higher performance on OOD than ID. 4. Why https://arxiv.org/abs/2104.08894 report much higher dimension for images? 5. Line 177: “equivalence mean that…” where is the equivalence? 6. Caption in Figure 6 is confusing: the title of the right-hand-side graph says it is the success rate of classification (higher the better) but the caption says it is the decrease of performance (lower the better). 7. What is the ID of the text generated by randomly sampling tokens from the vocabulary?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

The limitations are fairly elaborated.

Summary

The paper proposes using the intrinsic dimensionality of the manifold underlying the set of embeddings of a given text to detect AI-generated texts. Because the average intrinsic dimensionality of AI-generated texts is lower than that of natural language. It is found that the intrinsic dimensionality of different languages varies a lot.

Strengths

- The paper itself is novel - The finding that the intrinsic dimensionality of different languages varies a lot is interesting. - The method is robust in cross-domain and cross-model scenarios.

Weaknesses

- The paper does not discuss the reason for the difference of intrinsic dimensionality in different languages. - The reason why the method works isn't clearly understood.

Questions

- Intrinsic dimensionality is calculated in BERT/RoBERTa emebedding, I wonder if the *syntax of individual language* leads to the *difference of intrinsic dimensionality in different languages*. - Is there any analysis on the bad cases (incorrectly-classified cases)? which might help better understand the method.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

See the Weaknesses

Response acknowledged. Score remains the same (Accept)

This is to acknowledge that I have fully read both my fellow reviewers' feedback as well as the author/s' response to my own assessment. For the rebuttal period, the authors have provided some additional information for my concerns listed in my review (see weaknesses section). With this, I would like to summarize some points that the authors are strongly recommended add to the final paper in case of acceptance. This will ensure that any claims made on the paper are properly supported with evidence as well as discussed thoroughly. 1. In response to #1 weakness, the authors say that *"there are subtle differences that can be captured via numerical analysis of the topology of text embeddings"*. This is still vague, in my opinion. As suggested, it would be better to show a statistical test of difference on a number of runs with the ID method (say with different splits) for Table 2 or in Figure 6, with or without paraphrasers. 2. In response to #2 weakness, the authors say that *"we cannot state that our method will work in any domain shift situation."*, however when you read the paper, there are mentions of *“the PHD classifier is not influenced by domain transfer”* and *“we can expect the PHD classifier to be more robust to entirely new AI models”*. I suggest rephrasing these sentences instead, as it gives an inflated expectation for the classifier model. 3. Response to #4 weakness and Question #3 are good additions in their appropriate sections in the paper given provided more details. I was specifically looking for these in the paper. Nonetheless, I would like to thank the authors for their efforts and for being very detailed in their responses to my questions and concerns. My score will remain the same (7 - Accept) and would be happy to vouch for the paper to be accepted to the conference.

Acknowledgement of rebuttal

Thanks to the authors for addressing my concerns. I would encourage you to add the details provided in your response to the paper (as you have suggested). As already indicated with my initial scores, I believe this is solid work and therefore recommend acceptance.