Temporal In-Situ Compression of Scientific Floating Point Data with t-GLATE

Abstract

The need to cope with the huge amounts of data produced by today's HPC simulations has spurred the development of new lossy, error-bounded in-situ data compression methods. However, despite the time-continuous nature of HPC simulations, relatively few work has so far addressed temporal compression schemes. This article introduces t-GLATE, a compression algorithm designed for fast in-situ compression of time-dependent floating point data. t-GLATE compresses low-entropy exponents and high-entropy mantissas in two independent streams where mantissas are quantized according to a global look-up table. Temporal compression is achieved by storing the index differences rather than full look-up table indices for difference-frames. In experiments with our own implementations, several lossy in-situ compressors benefited from the inclusion of temporal compression schemes. We further compared t-GLATE to state-of-the-art compressors available as open source. In all test cases, t-GLATE compares favorably to other methods.