Intra To Inter: Towards Intra Prediction for Learning-Based Video Coders Using Optical Flow

Traditional video coders often rely on a block structure for transmission. Here each block is coded separately and sequentially and for each block the encoder can decide whether to use intra or inter prediction. This way, inter and intra prediction can be mixed within a single frame. This has advantages when new areas are uncovered, which were not present in the reference frame, and can hence not be predicted well. These areas are typically predicted using intra prediction. Currently much research goes into end-to-end-trained video coders which do not operate on a block level and typically use dense motion fields for inter prediction. There it is more difficult to incorporate intra prediction for uncovered regions. In this paper we propose a novel concept which enables us to reinterpret classical angular intra prediction in a way that we can transmit it as part of the dense motion field. We can save an average of 18% rate for the transmission of the motion vectors for the same quality of the prediction image.