Optimization problems pervade essentially every scientific discipline and\nindustry. Many such problems require finding a solution that maximizes the\nnumber of constraints satisfied. Often, these problems are particularly\ndifficult to solve because they belong to the NP-hard class, namely algorithms\nthat always find a solution in polynomial time are not known. Over the past\ndecades, research has focused on developing heuristic approaches that attempt\nto find an approximation to the solution. However, despite numerous research\nefforts, in many cases even approximations to the optimal solution are hard to\nfind, as the computational time for further refining a candidate solution grows\nexponentially with input size. Here, we show a non-combinatorial approach to\nhard optimization problems that achieves an exponential speed-up and finds\nbetter approximations than the current state-of-the-art. First, we map the\noptimization problem into a boolean circuit made of specially designed,\nself-organizing logic gates, which can be built with (non-quantum) electronic\ncomponents; the equilibrium points of the circuit represent the approximation\nto the problem at hand. Then, we solve its associated non-linear ordinary\ndifferential equations numerically, towards the equilibrium points. We\ndemonstrate this exponential gain by comparing a sequential MatLab\nimplementation of our solver with the winners of the 2016 Max-SAT competition\non a variety of hard optimization instances. We show empirical evidence that\nour solver scales linearly with the size of the problem, both in time and\nmemory, and argue that this property derives from the collective behavior of\nthe simulated physical circuit. Our approach can be applied to other types of\noptimization problems and the results presented here have far-reaching\nconsequences in many fields.\n