We investigate the set of (real Dolbeault classes of) balanced metrics $Θ$ on a balanced manifold $X$ with respect to which a torsion-free coherent sheaf $\mathcal{E}$ on $X$ is slope stable. We prove that the set of all such $[Θ] \in H^{n - 1,n - 1}(X,\mathbb{R})$ is an open convex cone locally defined by a finite number of linear inequalities. When $\mathcal{E}$ is a Hermitian vector bundle, the Kobayashi--Hitchin correspondence provides associated Hermitian Yang--Mills connections, which we show depend continuously on the metric, even around classes with respect to which $\mathcal{E}$ is only semi-stable. In this case, the holomorphic structure induced by the connection is the holomorphic structure of the associated graded object. The method relies on semi-stable perturbation techniques for geometric PDEs with a moment map interpretation and is quite versatile, and we hope that it can be used in other similar problems.