MANTRA: A Machine Learning reference lightcurve dataset for astronomical transient event recognition

We introduce Many ANnotated TRAnsients (MANTRA), an annotated data set of 4869 transient and 71207 non-transient object light curves built from the Catalina Real-time Transient Survey. We provide public access to this data set as a plain text file to facilitate standardized quantitative comparison of astronomical transient event recognition algorithms. Some of the classes included in the data set are: supernovae, cataclysmic variables, active galactic nuclei, high proper motion stars, blazars, and flares. As an example of the tasks that can be performed on the data set we experiment with multiple data preprocessing methods, feature selection techniques, and popular machine-learning algorithms (support vector machines, random forests, and neural networks). We assess quantitative performance in two classification tasks: binary (transient/non-transient) and eight-class classification. The best-performing algorithm in both tasks is the random forest classifier. It achieves an F1 score of 96.25% in the binary classification and 52.79% in the eight-class classification. For the eight-class classification, non-transients (96.83%) is the class with the highest F1 score, while the lowest corresponds to high proper motion stars (16.79%); for supernovae it achieves a value of 54.57%, close to the average across classes. The next release of MANTRA includes images and benchmarks with deep learning models.