Symbolic regression is the process of identifying mathematical expressions\nthat fit observed output from a black-box process. It is a discrete\noptimization problem generally believed to be NP-hard. Prior approaches to\nsolving the problem include neural-guided search (e.g. using reinforcement\nlearning) and genetic programming. In this work, we introduce a hybrid\nneural-guided/genetic programming approach to symbolic regression and other\ncombinatorial optimization problems. We propose a neural-guided component used\nto seed the starting population of a random restart genetic programming\ncomponent, gradually learning better starting populations. On a number of\ncommon benchmark tasks to recover underlying expressions from a dataset, our\nmethod recovers 65% more expressions than a recently published top-performing\nmodel using the same experimental setup. We demonstrate that running many\ngenetic programming generations without interdependence on the neural-guided\ncomponent performs better for symbolic regression than alternative formulations\nwhere the two are more strongly coupled. Finally, we introduce a new set of 22\nsymbolic regression benchmark problems with increased difficulty over existing\nbenchmarks. Source code is provided at\nwww.github.com/brendenpetersen/deep-symbolic-optimization.\n