Combating ChatGPT-Based Programming Test Cheating — An Evaluation Using Public Problems

The use of AI chatbots, such as ChatGPT, is increasingly popular across various domains, including education. While these tools offer valuable assistance to educators and students in teaching and learning activities, there is a growing concern about their potential misuse by students for cheating in assessments. This work aims to address the question of whether programming educators should be overly concerned about the misuse of ChatGPT by students. Our proposed approach consists of a two-step problem-solving process to explore the capabilities and limitations of ChatGPT in solving programming exercises. Our experiment results reveal that, despite ChatGPT's proficiency in generating solutions to programming problems, educators can strategically design authentic problems that challenge the tool, allowing for a more accurate evaluation of students' genuine academic performance.