Dense video captioning aims to localize and describe important events in\nuntrimmed videos. Existing methods mainly tackle this task by exploiting only\nvisual features, while completely neglecting the audio track. Only a few prior\nworks have utilized both modalities, yet they show poor results or demonstrate\nthe importance on a dataset with a specific domain. In this paper, we introduce\nBi-modal Transformer which generalizes the Transformer architecture for a\nbi-modal input. We show the effectiveness of the proposed model with audio and\nvisual modalities on the dense video captioning task, yet the module is capable\nof digesting any two modalities in a sequence-to-sequence task. We also show\nthat the pre-trained bi-modal encoder as a part of the bi-modal transformer can\nbe used as a feature extractor for a simple proposal generation module. The\nperformance is demonstrated on a challenging ActivityNet Captions dataset where\nour model achieves outstanding performance. The code is available:\nv-iashin.github.io/bmt\n