As large-scale language model pretraining pushes the state-of-the-art in text\ngeneration, recent work has turned to controlling attributes of the text such\nmodels generate. While modifying the pretrained models via fine-tuning remains\nthe popular approach, it incurs a significant computational cost and can be\ninfeasible due to lack of appropriate data. As an alternative, we propose\nMuCoCO -- a flexible and modular algorithm for controllable inference from\npretrained models. We formulate the decoding process as an optimization problem\nwhich allows for multiple attributes we aim to control to be easily\nincorporated as differentiable constraints to the optimization. By relaxing\nthis discrete optimization to a continuous one, we make use of Lagrangian\nmultipliers and gradient-descent based techniques to generate the desired text.\nWe evaluate our approach on controllable machine translation and style transfer\nwith multiple sentence-level attributes and observe significant improvements\nover baselines.\n