Using activation histograms to bound the number of affine regions in ReLU feed-forward neural networks

Several current bounds on the maximal number of affine regions of a ReLU\nfeed-forward neural network are special cases of the framework [1] which relies\non layer-wise activation histogram bounds. We analyze and partially solve a\nproblem in algebraic topology the solution of which would fully exploit this\nframework. Our partial solution already induces slightly tighter bounds and\nsuggests insight in how parameter initialization methods can affect the number\nof regions. Furthermore, we extend the framework to allow the composition of\nsubnetwork instead of layer-wise activation histogram bounds to reduce the\nnumber of required compositions which negatively affect the tightness of the\nresulting bound.\n