Variational approximations are increasingly based on gradient-based\noptimization of expectations estimated by sampling. Handling discrete latent\nvariables is then challenging because the sampling process is not\ndifferentiable. Continuous relaxations, such as the Gumbel-Softmax for\ncategorical distribution, enable gradient-based optimization, but do not define\na valid probability mass for discrete observations. In practice, selecting the\namount of relaxation is difficult and one needs to optimize an objective that\ndoes not align with the desired one, causing problems especially with models\nhaving strong meaningful priors. We provide an alternative differentiable\nreparameterization for categorical distribution by composing it as a mixture of\ndiscrete normalizing flows. It defines a proper discrete distribution, allows\ndirectly optimizing the evidence lower bound, and is less sensitive to the\nhyperparameter controlling relaxation.\n