A longstanding problem for deployed deep neural networks (DNNs) is their behavior in the face of unfamiliar inputs; specifically, these models tend to be overconfident and incorrect when encountering out-of-distribution (OOD) examples. In this work, we present a topological approach to characterizing OOD examples based on embeddings from DNNs. Our goal is to identify topological features that distinguish OOD inputs from in-distribution ones. Our experiments on benchmark datasets reveal that well-trained DNNs induce a topological simplification on training data, but not for OOD examples. More specifically, we find that the average lifetime (or persistence) of OOD examples is statistically longer than that of training or test examples. This indicates that DNNs struggle to induce topological simplification on unfamiliar inputs. Our empirical results provide novel evidence of topological simplification in realistic DNNs and lay the groundwork for topologically-informed OOD detection strategies.