The Llama 4 Herd: Architecture, Training, Evaluation, and Deployment Notes

This document consolidates publicly reported technical details about Metas Llama 4 model family. It summarizes (i) released variants (Scout and Maverick) and the broader herd context including the previewed Behemoth teacher model, (ii) architectural characteristics beyond a high-level MoE description covering routed/shared-expert structure, early-fusion multimodality, and long-context design elements reported for Scout (iRoPE and length generalization strategies), (iii) training disclosures spanning pre-training, mid-training for long-context extension, and post-training methodology (lightweight SFT, online RL, and lightweight DPO) as described in release materials, (iv) developer-reported benchmark results for both base and instruction-tuned checkpoints, and (v) practical deployment constraints observed across major serving environments, including provider-specific context limits and quantization packaging. The manuscript also summarizes licensing obligations relevant to redistribution and derivative naming, and reviews publicly described safeguards and evaluation practices. The goal is to provide a compact technical reference for researchers and practitioners who need precise, source-backed facts about Llama 4.

Paper

References (11)

01Llama 4 community license agreementLicense text
02The llama 4 herd: The beginning of a new era of natively multimodal ai innovationMeta AI Blog
03meta-llama/llama-4-maverick-17b-128e-instruct: Model card and license excerptHugging Face model card
04Meta’s llama 4 is now available on workers aiCloudflare Blog
05Meta’s llama license is still not open sourceOpen Source Initiative Blog
06Meta got caught gaming ai benchmarksThe Verge, 2025
07Meta releases two llama 4 ai modelsThe Verge
08Meta releases new ai model llama 4Reuters
09Llama 4: Model cards and prompt formatsLlama documentation
10meta-llama/llama-guard-4-12b: Model cardHugging Face model card
11Llama 4 models from meta now available in amazon bedrock serverlessAWS News Blog

Similar papers

© 2026 NYSGPT2525 LLC