PyMT5: multi-mode translation of natural language and Python code with transformers

Simultaneously modeling source code and natural language has many exciting\napplications in automated software development and understanding. Pursuant to\nachieving such technology, we introduce PyMT5, the Python method text-to-text\ntransfer transformer, which is trained to translate between all pairs of Python\nmethod feature combinations: a single model that can both predict whole methods\nfrom natural language documentation strings (docstrings) and summarize code\ninto docstrings of any common style. We present an analysis and modeling effort\nof a large-scale parallel corpus of 26 million Python methods and 7.7 million\nmethod-docstring pairs, demonstrating that for docstring and method generation,\nPyMT5 outperforms similarly-sized auto-regressive language models (GPT2) which\nwere English pre-trained or randomly initialized. On the CodeSearchNet test\nset, our best model predicts 92.1% syntactically correct method bodies,\nachieved a BLEU score of 8.59 for method generation and 16.3 for docstring\ngeneration (summarization), and achieved a ROUGE-L F-score of 24.8 for method\ngeneration and 36.7 for docstring generation.\n

Paper

Similar papers

© 2026 NYSGPT2525 LLC