To help understand the underlying mechanisms of neural networks (NNs), several groups have studied the number of linear regions <inline-formula> <tex-math notation="LaTeX">$\ell $ </tex-math></inline-formula> of piecewise linear (PwL) functions, generated by deep neural networks (DNN). In particular, they showed that <inline-formula> <tex-math notation="LaTeX">$\ell $ </tex-math></inline-formula> can grow exponentially with the number of network parameters <inline-formula> <tex-math notation="LaTeX">$p$ </tex-math></inline-formula>, a property often used to explain the advantages of deep over shallow NNs. Nonetheless, a dimension argument shows that DNNs cannot generate all PwL functions with <inline-formula> <tex-math notation="LaTeX">$\ell $ </tex-math></inline-formula> linear regions when <inline-formula> <tex-math notation="LaTeX">$\ell > p$ </tex-math></inline-formula>. It is thus natural to seek to characterize specific families of functions with <inline-formula> <tex-math notation="LaTeX">$\ell > p$ </tex-math></inline-formula> linear regions that can be constructed by DNNs. Iterated Function Systems (IFS) recursively construct a sequence of PwL functions <inline-formula> <tex-math notation="LaTeX">$F_{k}$ </tex-math></inline-formula> with a number of linear regions which is exponential in <inline-formula> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula>. We show that <inline-formula> <tex-math notation="LaTeX">$F_{k}$ </tex-math></inline-formula> can be generated by a NN using only <inline-formula> <tex-math notation="LaTeX">$\mathcal {O}(k)$ </tex-math></inline-formula> parameters. IFS are used extensively to generate natural-looking landscape textures in artificial images as well as for compression of natural images. The surprisingly good performance of this compression suggests that human visual system may lock in on self-similarities. The combination of this phenomenon with the capacity of DNNs to efficiently approximate IFS may contribute to the success of DNNs, particularly striking for image processing tasks.