Controlling bias in training datasets is vital for ensuring equal treatment,\nor parity, between different groups in downstream applications. A naive\nsolution is to transform the data so that it is statistically independent of\ngroup membership, but this may throw away too much information when a\nreasonable compromise between fairness and accuracy is desired. Another common\napproach is to limit the ability of a particular adversary who seeks to\nmaximize parity. Unfortunately, representations produced by adversarial\napproaches may still retain biases as their efficacy is tied to the complexity\nof the adversary used during training. To this end, we theoretically establish\nthat by limiting the mutual information between representations and protected\nattributes, we can assuredly control the parity of any downstream classifier.\nWe demonstrate an effective method for controlling parity through mutual\ninformation based on contrastive information estimators and show that they\noutperform approaches that rely on variational bounds based on complex\ngenerative models. We test our approach on UCI Adult and Heritage Health\ndatasets and demonstrate that our approach provides more informative\nrepresentations across a range of desired parity thresholds while providing\nstrong theoretical guarantees on the parity of any downstream algorithm.\n