Procedural models are being widely used to synthesize scenes for graphics,\ngaming, and to create (labeled) synthetic datasets for ML. In order to produce\nrealistic and diverse scenes, a number of parameters governing the procedural\nmodels have to be carefully tuned by experts. These parameters control both the\nstructure of scenes being generated (e.g. how many cars in the scene), as well\nas parameters which place objects in valid configurations. Meta-Sim aimed at\nautomatically tuning parameters given a target collection of real images in an\nunsupervised way. In Meta-Sim2, we aim to learn the scene structure in addition\nto parameters, which is a challenging problem due to its discrete nature.\nMeta-Sim2 proceeds by learning to sequentially sample rule expansions from a\ngiven probabilistic scene grammar. Due to the discrete nature of the problem,\nwe use Reinforcement Learning to train our model, and design a feature space\ndivergence between our synthesized and target images that is key to successful\ntraining. Experiments on a real driving dataset show that, without any\nsupervision, we can successfully learn to generate data that captures discrete\nstructural statistics of objects, such as their frequency, in real images. We\nalso show that this leads to downstream improvement in the performance of an\nobject detector trained on our generated dataset as opposed to other baseline\nsimulation methods. Project page:\nhttps://nv-tlabs.github.io/meta-sim-structure/.\n