HYBRID TRANSFORM IN VIDEO ENCODING AND DECODING

Blocks of a frame of a video stream can be encoded using intra prediction followed by transforming the generated residual blocks. In some cases, coding efficiency can be improved by subdividing large blocks into smaller sub-blocks. Transform modes may be associated with at least one intra prediction mode so as to identify one or more transform modes for the block or its sub-blocks. A scan mode can be identified based on the transform mode. Desirably, each of a plurality of probability models is associated with a transform mode and a block size such that the appropriate model(s) may be selected for a block or its sub-blocks. Each probability model may be associated with only one transform mode and only one block size or may be associated with more than one transform mode and/or more than one block size and may be updated on-the-fly during encoding of a video stream.