Perfect Gibbs Sampling of Order Constrained Non-IID Ordered Random Variates with Application to Bayesian Principal Components Analysis

Order statistics arising from $m$ independent but not identically distributed random variables are typically constructed by arranging some $X_{1}, X_{2}, \ldots, X_{m}$, with $X_{i}$ having distribution function $F_{i}(x)$, in increasing order denoted as $X_{(1)} \leq X_{(2)} \leq \ldots \leq X_{(m)}$. In this case, $X_{(i)}$ is not necessarily associated with $F_{i}(x)$. Assuming one can simulate values from each distribution, one can generate such ``non-iid order statistics by simulating $X_{i}$ from $F_{i}$, for $i=1,2,\ldots, m$, and simply putting them in order. In this paper, we consider the problem of simulating ordered values $X_{(1)}, X_{(2)}, \ldots, X_{(m)}$ such that the marginal distribution of $X_{(i)}$ is $F_{i}(x)$. This problem arises in Bayesian principal components analysis (BPCA) where the $X_{i}$ are ordered eigenvalues that are a posteriori independent but not identically distributed. In this paper, we propose a novel {\emph{coupling-from-the-past}} algorithm to ``perfectly (up to computable order of accuracy) simulate such order-constrained non-iid order statistics. We demonstrate the effectiveness of our approach for several examples, including the BPCA problem.