Bayesian optimization (BO) is a powerful approach for optimizing black-box,\nexpensive-to-evaluate functions. To enable a flexible trade-off between the\ncost and accuracy, many applications allow the function to be evaluated at\ndifferent fidelities. In order to reduce the optimization cost while maximizing\nthe benefit-cost ratio, in this paper, we propose Batch Multi-fidelity Bayesian\nOptimization with Deep Auto-Regressive Networks (BMBO-DARN). We use a set of\nBayesian neural networks to construct a fully auto-regressive model, which is\nexpressive enough to capture strong yet complex relationships across all the\nfidelities, so as to improve the surrogate learning and optimization\nperformance. Furthermore, to enhance the quality and diversity of queries, we\ndevelop a simple yet efficient batch querying method, without any combinatorial\nsearch over the fidelities. We propose a batch acquisition function based on\nMax-value Entropy Search (MES) principle, which penalizes highly correlated\nqueries and encourages diversity. We use posterior samples and moment matching\nto fulfill efficient computation of the acquisition function and conduct\nalternating optimization over every fidelity-input pair, which guarantees an\nimprovement at each step. We demonstrate the advantage of our approach on four\nreal-world hyperparameter optimization applications.\n