Automated Machine Learning (AutoML) has gained increasing success on tabular\ndata in recent years. However, processing unstructured data like text is a\nchallenge and not widely supported by open-source AutoML tools. This work\ncompares three manually created text representations and text embeddings\nautomatically created by AutoML tools. Our benchmark includes four popular\nopen-source AutoML tools and eight datasets for text classification purposes.\nThe results show that straightforward text representations perform better than\nAutoML tools with automatically created text embeddings.\n