Difference-of-Convex Algorithm (DCA) is a well-known nonconvex optimization\nalgorithm for minimizing a nonconvex function that can be expressed as the\ndifference of two convex ones. Many famous existing optimization algorithms,\nsuch as SGD and proximal point methods, can be viewed as special DCAs with\nspecific DC decompositions, making it a powerful framework for optimization. On\nthe other hand, shortcuts are a key architectural feature in modern deep neural\nnetworks, facilitating both training and optimization. We showed that the\nshortcut neural network gradient can be obtained by applying DCA to vanilla\nneural networks, networks without shortcut connections. Therefore, from the\nperspective of DCA, we can better understand the effectiveness of networks with\nshortcuts. Moreover, we proposed a new architecture called NegNet that does not\nfit the previous interpretation but performs on par with ResNet and can be\nincluded in the DCA framework.\n