Over the past several years, there have been many studies demonstrating the ability of deep neural networks to identify phase transitions in many physical systems, notably in classical statistical physics systems. One often finds that the prediction of deep learning methods trained on many ensembles below and above the critical temperature T_{c} behaves similarly to an order parameter, and this analogy has been successfully used to locate T_{c} and estimate universal critical exponents. In this work, we pay particular attention to the ability of a convolutional neural network to capture these critical parameters for the two-dimensional Ising model when the network is trained on configurations at T=0 and T=∞ only. We directly compare its output to the same network trained at multiple temperatures below and above T_{c} to gain understanding of how this extreme restriction of training data can impact a neural network's ability to classify phases. We find that the network trained on two temperatures is still able to identify T_{c} and ν, while the extraction of γ becomes more challenging.