DEVELOPING A PROGRAMMING LANGUAGE MODEL FOR MACHINE LEARNING TASKS

A method develops a programming language model for machine learning tasks. The method includes adjusting a token list to include a language token used by a tokenizer for a pretrained language model. The pretrained language model includes a set of layers. The set of layers includes a set of initial layers, an embedding layer, and an output layer. The method further includes performing an output layer modification of the output layer to replace the output vector with the embedding vector. The method further includes freezing the set of initial layers to generate a set of frozen layers of the pretrained language model that do not update during training. The method further includes training the pretrained language model using the language token, the output layer modification, and the set of frozen layers to form a fine-tuned model from the pretrained language model.