METHOD, APPARATUS AND SYSTEM FOR ENCODING AND DECODING THE SIGNIFICANCE MAP FOR RESIDUAL COEFFICIENTS OF A TRANSFORM UNIT

Disclosed is a method of decoding residual coefficients of a transform unit from a bitstream of video data. The method receives the transform unit (1500) from the bitstream of video data in which the transform unit has upper (1503) and lower (1502) level square layers. The upper level layer represents a square arrangement of at most four significant coefficient group flags with each of the significant coefficient group flags representing a non-overlapping region of the lower level layer. The method determines determining the significant coefficient group flags of the square upper level layer for the received transform unit, and determines values of residual coefficients of the square lower layer according to the determined significant coefficient group flags to decode the transform unit of the bitstream of video data. Corresponding encoding methods are also disclosed.