Learning interpretable and human-controllable representations that uncover\nfactors of variation in data remains an ongoing key challenge in representation\nlearning. We investigate learning group-disentangled representations for groups\nof factors with weak supervision. Existing techniques to address this challenge\nmerely constrain the approximate posterior by averaging over observations of a\nshared group. As a result, observations with a common set of variations are\nencoded to distinct latent representations, reducing their capacity to\ndisentangle and generalize to downstream tasks. In contrast to previous works,\nwe propose GroupVAE, a simple yet effective Kullback-Leibler (KL)\ndivergence-based regularization across shared latent representations to enforce\nconsistent and disentangled representations. We conduct a thorough evaluation\nand demonstrate that our GroupVAE significantly improves group disentanglement.\nFurther, we demonstrate that learning group-disentangled representations\nimprove upon downstream tasks, including fair classification and 3D\nshape-related tasks such as reconstruction, classification, and transfer\nlearning, and is competitive to supervised methods.\n