LiVeAction: a Lightweight, Versatile, and Asymmetric Neural Codec Design for Real-time Operation

Modern sensors generate rich, high-fidelity data, yet applications operating on wearable or remote sensing devices remain constrained by bandwidth and power budgets. Standardized JPEG and MPEG codecs achieve efficient trade-offs between bit-rate and quality for audio, images, and video, but have limited applicability for machine-perception tasks and non-traditional modalities. Recent generative neural codecs, or tokenizers, require millions of samples to train and are impractical for resource-constrained environments due to their large DNN-based analysis transforms. We introduce a Lightweight, Versatile, and Asymmetric neural codec design (LiVe-Action), focusing on two key ideas. (1) To reduce the complexity of the encoder, we impose an FFT-like structure and reduce the overall size and depth of the neural-network-based analysis transform. (2) To reduce the training burden and increase versatility for other types of signals, we remove adversarial and perceptual losses and focus on MSE loss with a simplified rate penalty. We release our code, experiments, and python library at https://github.com/ut-sysml/liveaction.