CodeMark: Contextual and Natural Watermarking for Tracing Code Snippet Provenance

Abstract

Determining the origins of code snippets has gained increasing attention due to the popularity of large language models and the concern about their misuse in generating unlicensed or malicious code. Watermarking is considered a working solution for tracing code snippet provenance. However, source code watermarking requires more stringent and intricate rules than natural language or software watermarking, since one needs to ensure both readability and functionality of the watermarked code snippets. To this end, we propose a novel watermarking system CodeMark, featured by variable renaming as the key. Surrounding variable renaming, several challenges emerge such as determining renaming candidates, defining the variable context, and providing diverse variable substitutes, etc. The design of CodeMark conquers these challenges by an end-to-end learning system, which interprets the code context through Graph Neural Networks (GNNs) and generates natural substitutes fitting the context by distilling from CodeBERT. Experiments illustrate that CodeMark surpasses the state-of-the-art watermarking systems in terms of watermarking requirements.