In this work we study the problem of measuring the fairness of a machine\nlearning model under noisy information. Focusing on group fairness metrics, we\ninvestigate the particular but common situation when the evaluation requires\ncontrolling for the confounding effect of covariate variables. In a practical\nsetting, we might not be able to jointly observe the covariate and group\ninformation, and a standard workaround is to then use proxies for one or more\nof these variables. Prior works have demonstrated the challenges with using a\nproxy for sensitive attributes, and strong independence assumptions are needed\nto provide guarantees on the accuracy of the noisy estimates. In contrast, in\nthis work we study using a proxy for the covariate variable and present a\ntheoretical analysis that aims to characterize weaker conditions under which\naccurate fairness evaluation is possible.\n Furthermore, our theory identifies potential sources of errors and decouples\nthem into two interpretable parts $\\gamma$ and $\\epsilon$. The first part\n$\\gamma$ depends solely on the performance of the proxy such as precision and\nrecall, whereas the second part $\\epsilon$ captures correlations between all\nthe variables of interest. We show that in many scenarios the error in the\nestimates is dominated by $\\gamma$ via a linear dependence, whereas the\ndependence on the correlations $\\epsilon$ only constitutes a lower order term.\nAs a result we expand the understanding of scenarios where measuring model\nfairness via proxies can be an effective approach. Finally, we compare, via\nsimulations, the theoretical upper-bounds to the distribution of simulated\nestimation errors and show that assuming some structure on the data, even weak,\nis key to significantly improve both theoretical guarantees and empirical\nresults.\n