In multilingual neural machine translation, it has been shown that sharing a\nsingle translation model between multiple languages can achieve competitive\nperformance, sometimes even leading to performance gains over bilingually\ntrained models. However, these improvements are not uniform; often multilingual\nparameter sharing results in a decrease in accuracy due to translation models\nnot being able to accommodate different languages in their limited parameter\nspace. In this work, we examine parameter sharing techniques that strike a\nhappy medium between full sharing and individual training, specifically\nfocusing on the self-attentional Transformer model. We find that the full\nparameter sharing approach leads to increases in BLEU scores mainly when the\ntarget languages are from a similar language family. However, even in the case\nwhere target languages are from different families where full parameter sharing\nleads to a noticeable drop in BLEU scores, our proposed methods for partial\nsharing of parameters can lead to substantial improvements in translation\naccuracy.\n