ERGO-ML: The assembly histories of HSC galaxy images via invertible neural networks, contrastive learning, and cosmological simulations

In this paper of ERGO-ML (Extracting Reality from Galaxy Observables with Machine Learning), we develop a model that infers the merger/assembly histories of galaxies directly from optical images. We apply the self-supervised contrastive learning framework NNCLR on realistic HSC mock images (g,r,i - bands) produced from galaxies simulated within the TNG50 and TNG100 runs of the IllustrisTNG project and with stellar masses of $10^{9-12} \rm {M}_{\odot }$. The resulting representation is then used as conditional input for a cINN to gain posteriors for merger/assembly statistics, particularly the lookback time and stellar mass of the last major merger and the fraction of ex-situ stars. We achieve good accuracy in inferring the stellar ex-situ fraction (≤± 10percnt for 80percnt of the test sample). The information content about the lookback time is, instead, limited. We also successfully apply the TNG-trained model to simulated mocks from the EAGLE simulation, demonstrating that our model is applicable outside of the TNG domain. We hence use our simulation-based model to infer aspects of the history of observed galaxies, in particular for HSC images that are close to the domain of TNG ones. We recover the trend of increasing ex-situ stellar fraction with stellar mass and more spherical morphology, but we also identify a discrepancy between TNG and HSC: on average, observed galaxies generally exhibit lower ex-situ fractions. Despite challenges such as information loss and domain shifts, our results demonstrate the feasibility of extracting the merger past of galaxies from their optical images.