Bot or Human? Detecting ChatGPT Imposters with A Single Question

Summary

The paper evaluates a number of tasks on several LLMs and on humans that are a priori selected and experimentally confirmed (in most instances) to be either very hard for LLMs and easy for humans or the other way round. The main motivation put forward by the authors is to identify ways to perform one-shot classification between human and artificial conversational agents.

Rating

6

Confidence

4

Ethics flag

1

Reasons to accept

- The paper is based on a simple yet relevant idea that the authors investigate in a simple yet effective way

Reasons to reject

- The impact of the work is not sufficiently discussed. The authors write that "overcoming [the poor level of GPT-4 ASCII-art related capabilities] would necessitate significant investment, potentially amounting to millions of dollars, for developing a dedicated ASCII art reasoning model". I am not convinced by this statement at all. GPT-4 is very good at multiple tasks for which no "dedicated reasoning model" was developed. And the "millions of dollars" ballpark is not motivated at all. On the other hand, the authors do not mention the fact that a malevolent LLM practitioner might try to make their LLM bad at memorization and computation tasks, in order to behave in a more human-like fashion. - The paper needs to be proofread. It is regrettable for a paper to be submitted with typos like the following, all easily detected by any good spell checker (including Microsoft Word or… ChatGPT): - page 3 "another line of study model this problem" ("models") - page 4 "an example of LLMs and humans output" ("an example of LLM and human output") - page 4 "As we can seen" ("see") - page 4 "This first stage of our proposed method is convert" ("is to convert") - page 5 "The next step is perform" ("is to perform") - page 6 "Below is and example" ("an") - page 6 "it require a good memorization" ("requires") - page 8 "10 seconds time limit" ("10-second time limit", with hyphen but no -s) - page 9 "develope" ("develop") - page 9 "possivle" ("possible") - page 9 "to Combine" ("to combine") - some citations should have been made using \citep and not \citet (examples: all citations in section 5.2, "Awla et al. (2022); Guerar et al. (2021)" on page 9)

Questions to authors

- The most interesting tasks being ASCII-art-related, how do you relate your results on these tasks with the ArtPrompt paper by Jiang et al. 2024 (published after the submission deadline), where the authors seem to show that LLMs do understand ASCII-art to some extent?

Summary

This paper introduces a novel method to detect whether a participant in a conversation is a human or a bot. Specifically, they evaluated various question-answer pairs on both humans and LLMs. These were single QA pairs that included questions about counting, substitution, random editing, ASCII art reasoning, memorization, and computation. The questions are designed around things that the LLMs are good at (e.g., memorization and computation) and bad at (i.e., other tasks). Overall, the paper finds that ascii-related questions can accurately be used to detect if someone is a bot or not. Specifically, only 1% of questions were accurately answered on the ASCII rotate questions for some models, but 97% were accurate for humans. Hence, these systems can potentially be used by companies and online services to mitigate bot accounts.

Rating

6

Confidence

3

Ethics flag

1

Reasons to accept

- Overall, this is a really unique study that explores ways to detect bots in the wild (e.g., online service providers) so they can't cause issues. I- definitely want to highlight the creativity of the ASCII solution. This is really out-of-the-box thinking. - The paper is generally well-written.

Reasons to reject

- The paper would be improved if there was a broader discussion about frameworks for how new questions can be generated as LLMs overcome these limitations. If the questions require manual intervention, they will not necessarily scale at the speed needed to mitigate attacks. While the findings are exceptionally creative, it does seem a bit early in the analysis to understand the impact of the findings. Could there be a general automated way to generate these questions that target weaknesses? If so, how would it be done? At a minimum, if substantially more work is required to do automated question-answering, this should be mentioned as a limitation. - More description about how accuracy was calculated. There are also some general questions that I have, e.g., why median accuracy vs. mean accuracy? How much did the accuracy vary trial-to-trial? - [Minor Weakness] I'm not entirely sure about the usefulness of this system in the application for VLLMs. Particularly, what would stop a bot from screenshotting a website, performing a rotation on the image, then converting back to ascii art? This is possible via GPT4 online interface based on superficial things I tested (e.g., it generates code that converts an image to ASCII and creates code that can perform rotations on the image). I assume a similar thing can be done for matching to images, e.g., converting the ASCII art to an image and then doing image-to-image matching.

Questions to authors

- Why median accuracy vs. mean accuracy? - How much did the accuracy vary from trial to trial?

Summary

This paper propose a new CAPTCHAs methods to detect whether the response is from human or LLMs via asking a question that is either easy to LLM but hard to human or easy to human but hard to LLM. The researcher identify a few patterns of such questions that can lead to distinguished performance of human and LLMs. The proposed questions are checked against human users and a comprehensive set of LLMs. The results show that these questions can effectively distinguish the LLMs and Humans based on the accuracy of their response;

Rating

5

Confidence

4

Ethics flag

1

Reasons to accept

1. A simple and clean design to capture the different behaviors from human users and LLM via asking a question that will lead to completely different response from the two groups. 2. Details of the question design are clearly explained which makes it easy to replicate the dataset by following reseaches. 3. The benchmark is also valuable to check LLM's capability on the founda-mental skills that is easy for human

Reasons to reject

1. It is questionable on the time-wise effectiveness of this approach to distinguish LLM from human as we don't know whether the challenges faced by LLMs now will still remain when newer version of LLMs come out. Some of the challenges seem to be easily addressed by providing more in-domain data to the model which makes the trick easily broken. 2. For the question that are easy to LLM but hard to human, it is really hard to control whether the human will actually answer the questions without the help of computational tool or AI tool during the actual CAPTCHA check. Therefore, it is hard to really distinguish humans from LLMs by simply checking whether the memory type questions are answered correctly;

Comment

Thanks for your response. This clarifies many of my questions. Regarding "The identification of the question class requires substantial research and experimentation," this should be discussed more as a limitation, particularly if methods can easily be trained specifically to overcome these tasks that were presented.

Response to Author's Rebuttal

Thanks for authors' responses. To me, my concern on the time-wise effectiveness of these set of questions still remain. Especially, the design of the questions heavily rely on manual effort, which makes it hard to scale up, whiich makes it hard to be deployed as a reliable Recaptha application. However, I do appreciate the value of the benchmark to study the distinguished behavior of humans and LLMs to the same set of questions and how to address the LLM's limited capability on them. I will slightly increase my score, but I personally would still not favor this method as a RECAPTCHA implementation.