YOLoC: DeploY Large-Scale Neural Network by ROM-based Computing-in-Memory using ResiduaL Branch on a Chip

Computing-in-memory (CiM) is a promising technique to achieve high energy efficiency in data-intensive matrix-vector multiplication (MVM) by relieving the memory bottleneck. Unfortunately, due to the limited SRAM capacity, existing SRAMbased CiM needs to reload the weights from DRAM in large-scale networks. This undesired fact weakens the energy efficiency significantly. This work, for the first time, proposes the concept, design, and optimization of computing-in-ROM to achieve much higher on-chip memory capacity, and thus less DRAM access and lower energy consumption. Furthermore, to support different computing scenarios with varying weights, a weight fine-tune technique, namely Residual Branch (ReBranch), is also proposed. ReBranch combines ROM-CiM and assisting SRAM-CiM to achieve high versatility. YOLoC, a ReBranch-assisted ROM-CiM framework for object detection is presented and evaluated. With the same area in 28 nm CMOS, YOLoC for several datasets has shown significant energy efficiency improvement by 14.8 x for YOLO (DarkNet-19) and 4.8x for ResNet-18, with $\gt 8 \%$ latency overhead and almost no mean average precision (mAP) loss ($-0.5\% \sim+0.2 \%$), compared with the fully SRAM-based CiM.CCS Concepts• Hardware $\sim$ Integrated circuits $\sim$ Semiconductor memory $\sim$ Readonly memory • Computing methodologies $\sim$ Artificial intelligence $\sim$ Computer vision $\sim$ Computer vision problems $\sim$ Object detection