Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Models

Targeted syntactic evaluations have demonstrated the ability of language\nmodels to perform subject-verb agreement given difficult contexts. To elucidate\nthe mechanisms by which the models accomplish this behavior, this study applies\ncausal mediation analysis to pre-trained neural language models. We investigate\nthe magnitude of models' preferences for grammatical inflections, as well as\nwhether neurons process subject-verb agreement similarly across sentences with\ndifferent syntactic structures. We uncover similarities and differences across\narchitectures and model sizes -- notably, that larger models do not necessarily\nlearn stronger preferences. We also observe two distinct mechanisms for\nproducing subject-verb agreement depending on the syntactic structure of the\ninput sentence. Finally, we find that language models rely on similar sets of\nneurons when given sentences with similar syntactic structure.\n