Speed and Conversational Large Language Models: Not All Is About Tokens per Second

The speed of open-weights large language models (LLMs) and its dependency on the task at hand, when run on GPUs, is studied to present a comparative analysis of the speed of the most popular open LLMs.

cs.CL cs.AI Computer Science

View graph Source

Paper

Similar papers

Speed and Conversational Large Language Models: Not All Is About Tokens per SecondJavier Conde, Miguel Ángel Pesquera González, Pedro Reviriego et al.2024 · Computer In Library

Deploying Open-Source Large Language Models: A performance AnalysisYannis Bendi-Ouis, Dan Dutartre, Xavier Hinaut2024 · arXiv (Cornell University) · 6 citations In Library

Computational Bottlenecks of Training Small-scale Large Language ModelsSaleh Ashkboos, Iman Mirzadeh, Keivan Alizadeh et al.2024 · arXiv (Cornell University) · 4 citations In Library

All You Should Know About Large Language Models (LLMs)Abadila Alaktif, Meriyem Chergui, Imane Daoudi et al.2024 · International Conference on Advanced Communication Technologies and Networking · 2 citations In Library

Speed Always Wins: A Survey on Efficient Architectures for Large Language ModelsWeigao Sun, Jiaxi Hu, Yucheng Zhou et al.2025 · arXiv.org · 17 citations In Library

Hardware Acceleration of LLMs: A comprehensive survey and comparisonNikoletta Koilia, C. Kachris2024 · arXiv.org · 16 citations In Library

Acceleration of Large Language Models with Emerging Compute and Communication TechnologyJoerg C. Schefold, Sumit K. Mandal, Biresh Kumar Joardar2025 · Coins · 0 citations In Library

LLMPerf: GPU Performance Modeling meets Large Language ModelsMinh-Khoi Nguyen-Nhat, H. Do, H. Le et al.2024 · IEEE/ACM International Symposium on Modeling, Analysis, and Simulation On Computer and Telecommunication Systems · 3 citations In Library

Transformer-Lite: High-efficiency Deployment of Large Language Models on Mobile Phone GPUsLuchang Li, Sheng Qian, Jie Lu et al.2024 · arXiv (Cornell University) · 24 citations In Library

A Survey on Hardware Accelerators for Large Language ModelsChristoforos Kachris2024 · arXiv (Cornell University) · 59 citations In Library

Source of record

arXiv

44B

Welcome.

This panel sticks with you. Pick anything below and it opens right beside it, so you can dig through 60,000-plus records without ever losing your spot here.

Travel 44B

LibraryPapers, policy, standards, statute LabsEvery organization building AI ModelsIntelligence, price, and speed BenchmarksThe evaluation catalog IncidentsAI failures and harms SearchOne field across everything DashboardThe state of AI accountability in NY 44B RegistryThe Art. 44-B compliance portal