General-purpose pretrained sentence encoders such as BERT are not ideal for\nreal-world conversational AI applications; they are computationally heavy,\nslow, and expensive to train. We propose ConveRT (Conversational\nRepresentations from Transformers), a pretraining framework for conversational\ntasks satisfying all the following requirements: it is effective, affordable,\nand quick to train. We pretrain using a retrieval-based response selection\ntask, effectively leveraging quantization and subword-level parameterization in\nthe dual encoder to build a lightweight memory- and energy-efficient model. We\nshow that ConveRT achieves state-of-the-art performance across widely\nestablished response selection tasks. We also demonstrate that the use of\nextended dialog history as context yields further performance gains. Finally,\nwe show that pretrained representations from the proposed encoder can be\ntransferred to the intent classification task, yielding strong results across\nthree diverse data sets. ConveRT trains substantially faster than standard\nsentence encoders or previous state-of-the-art dual encoders. With its reduced\nsize and superior performance, we believe this model promises wider portability\nand scalability for Conversational AI applications.\n