The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits

End-to-end pipeline for training ternary-weight ({-1, 0, +1}) convolutional neural networks in PyTorch and deploying them as multiply-free inference engines on ESP32-S3 microcontrollers. Validated on MNIST (98.79% accuracy, -0.23 pp drop from FP32) and CIFAR-10 (69.02% accuracy, -6.50 pp drop) with 93.75% weight memory compression and zero multiplications in conv/linear layers. The inference engine is fully dynamic — it reads model dimensions from the binary format at runtime, supporting any input shape without recompilation. Includes PyTorch training scripts, custom .vbn binary export format, zero-dependency C++ desktop engine, and Arduino sketch for ESP32-S3.

Paper

Similar papers

© 2026 NYSGPT2525 LLC