In the language domain, as in other domains, neural explainability takes an\never more important role, with feature attribution methods on the forefront.\nMany such methods require considerable computational resources and expert\nknowledge about implementation details and parameter choices. To facilitate\nresearch, we present Thermostat which consists of a large collection of model\nexplanations and accompanying analysis tools. Thermostat allows easy access to\nover 200k explanations for the decisions of prominent state-of-the-art models\nspanning across different NLP tasks, generated with multiple explainers. The\ndataset took over 10k GPU hours (> one year) to compile; compute time that the\ncommunity now saves. The accompanying software tools allow to analyse\nexplanations instance-wise but also accumulatively on corpus level. Users can\ninvestigate and compare models, datasets and explainers without the need to\norchestrate implementation details. Thermostat is fully open source,\ndemocratizes explainability research in the language domain, circumvents\nredundant computations and increases comparability and replicability.\n