Relational regularized autoencoder (RAE) is a framework to learn the\ndistribution of data by minimizing a reconstruction loss together with a\nrelational regularization on the latent space. A recent attempt to reduce the\ninner discrepancy between the prior and aggregated posterior distributions is\nto incorporate sliced fused Gromov-Wasserstein (SFG) between these\ndistributions. That approach has a weakness since it treats every slicing\ndirection similarly, meanwhile several directions are not useful for the\ndiscriminative task. To improve the discrepancy and consequently the relational\nregularization, we propose a new relational discrepancy, named spherical sliced\nfused Gromov Wasserstein (SSFG), that can find an important area of projections\ncharacterized by a von Mises-Fisher distribution. Then, we introduce two\nvariants of SSFG to improve its performance. The first variant, named mixture\nspherical sliced fused Gromov Wasserstein (MSSFG), replaces the vMF\ndistribution by a mixture of von Mises-Fisher distributions to capture multiple\nimportant areas of directions that are far from each other. The second variant,\nnamed power spherical sliced fused Gromov Wasserstein (PSSFG), replaces the vMF\ndistribution by a power spherical distribution to improve the sampling time in\nhigh dimension settings. We then apply the new discrepancies to the RAE\nframework to achieve its new variants. Finally, we conduct extensive\nexperiments to show that the new proposed autoencoders have favorable\nperformance in learning latent manifold structure, image generation, and\nreconstruction.\n