We introduce the Byte Latent Transformer (BLT), a new byte-level LLM\narchitecture that, for the first time, matches tokenization-based LLM\nperformance at scale with significant improvements in inference efficiency and\nrobustness. BLT encodes bytes into dynamically sized patches, which serve as\nthe primary units of computation. Patches are segmented based on the entropy of\nthe next byte, allocating more compute and model capacity where increased data\ncomplexity demands it. We present the first FLOP controlled scaling study of\nbyte-level models up to 8B parameters and 4T training bytes. Our results\ndemonstrate the feasibility of scaling models trained on raw bytes without a\nfixed vocabulary. Both training and inference efficiency improve due to\ndynamically selecting long patches when data is predictable, along with\nqualitative improvements on reasoning and long tail generalization. Overall,\nfor fixed inference costs, BLT shows significantly better scaling than\ntokenization-based models, by simultaneously growing both patch and model size.\n