We present a randomized controlled trial for a model-in-the-loop regression\ntask, with the goal of measuring the extent to which (1) good explanations of\nmodel predictions increase human accuracy, and (2) faulty explanations decrease\nhuman trust in the model. We study explanations based on visual saliency in an\nimage-based age prediction task for which humans and learned models are\nindividually capable but not highly proficient and frequently disagree. Our\nexperimental design separates model quality from explanation quality, and makes\nit possible to compare treatments involving a variety of explanations of\nvarying levels of quality. We find that presenting model predictions improves\nhuman accuracy. However, visual explanations of various kinds fail to\nsignificantly alter human accuracy or trust in the model - regardless of\nwhether explanations characterize an accurate model, an inaccurate one, or are\ngenerated randomly and independently of the input image. These findings suggest\nthe need for greater evaluation of explanations in downstream decision making\ntasks, better design-based tools for presenting explanations to users, and\nbetter approaches for generating explanations.\n