Privacy and Byzantine resilience (BR) are two crucial requirements of\nmodern-day distributed machine learning. The two concepts have been extensively\nstudied individually but the question of how to combine them effectively\nremains unanswered. This paper contributes to addressing this question by\nstudying the extent to which the distributed SGD algorithm, in the standard\nparameter-server architecture, can learn an accurate model despite (a) a\nfraction of the workers being malicious (Byzantine), and (b) the other\nfraction, whilst being honest, providing noisy information to the server to\nensure differential privacy (DP). We first observe that the integration of\nstandard practices in DP and BR is not straightforward. In fact, we show that\nmany existing results on the convergence of distributed SGD under Byzantine\nfaults, especially those relying on $(\\alpha,f)$-Byzantine resilience, are\nrendered invalid when honest workers enforce DP. To circumvent this\nshortcoming, we revisit the theory of $(\\alpha,f)$-BR to obtain an approximate\nconvergence guarantee. Our analysis provides key insights on how to improve\nthis guarantee through hyperparameter optimization. Essentially, our\ntheoretical and empirical results show that (1) an imprudent combination of\nstandard approaches to DP and BR might be fruitless, but (2) by carefully\nre-tuning the learning algorithm, we can obtain reasonable learning accuracy\nwhile simultaneously guaranteeing DP and BR.\n