A 3.55 mJ/frame Energy-efficient Mixed-Transformer based Semantic Segmentation Accelerator for Mobile Devices

Abstract

An energy-efficient semantic segmentation (SS) processor, achieving 3.55 mJ/frame system energy efficiency, is proposed. To address the challenges posed by Mixed Transformer (MiT)-based SS, including high external memory bandwidth requirement and large on-chip memory footprint, we introduce a novel compression method called Chunk-based Bit Plane Compression (CBPC). CBPC leverages the high inter-token locality of feature maps in MiT-based SS, along with the robustness and compression ratio variations based on bit position to achieve a high compression ratio. To support CBPC, we propose an area and power-efficient CBPC encoder/decoder. In addition, a Similar Token Coarse Skipping (STCS) Core is proposed for high throughput. It enables row-wise clock gating and array-wise coarse skipping to reduce redundant computation. By removing redundant computation, the processor achieves higher throughput and lower computation power. The proposed processor reduces 67.6% of EMA power and accomplishes 19.24 TOPS/W core energy efficiency. The proposed processor achieves 44.3% higher system energy efficiency than the previous processors.