The explosive arrival of OpenAI's ChatGPT has fueled the globalization of\nlarge language model (LLM), which consists of billions of pretrained parameters\nthat embodies the aspects of syntax and semantics. HyperAccel introduces\nlatency processing unit (LPU), a latency-optimized and highly scalable\nprocessor architecture for the acceleration of LLM inference. LPU perfectly\nbalances the memory bandwidth and compute logic with streamlined dataflow to\nmaximize performance and efficiency. LPU is equipped with expandable\nsynchronization link (ESL) that hides data synchronization latency between\nmultiple LPUs. HyperDex complements LPU as an intuitive software framework to\nrun LLM applications. LPU achieves 1.25 ms/token and 20.9 ms/token for 1.3B and\n66B model, respectively, which is 2.09x and 1.37x faster than the GPU. LPU,\nsynthesized using Samsung 4nm process, has total area of 0.824 mm2 and power\nconsumption of 284.31 mW. LPU-based servers achieve 1.33x and 1.32x energy\nefficiency over NVIDIA H100 and L4 servers, respectively.\n